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ABSTRACT 



In March 2000, thirteen grants were awarded as part of the 
Individual Training Account /Eligible Training Provider ( ITA/ETP) 
Demonstration. In summer and fall of 2000, the grant recipients' activities 
were subjected to an interim evaluation. Site visits were made to each 
grantee to determine what ITA policies and practices were being formulated, 
how ETP lists were being assembled, and what information was available in the 
consumer report systems about eligible programs from which customers can 
choose a training provider. The following were among the interim evaluation's 
conclusions: (1) although flexibility is clearly being embraced by all 

grantees, it is causing some confusion and uncertainty; (2) for a variety of 
reasons, a substantial drop-off in the number of persons entering training is 
predicted; (3) sites are maintaining a strong customer focus in their 
approach to the Workforce Investment Act's three service levels; (4) customer 
choice, informed by good information, is clearly apparent in how sites are 
working with customers to help them select training programs and vendors; (5) 
frontline staff are generally playing roles that support informed choice; and 
(6) the underdeveloped state of most consumer report systems has meant that 
most customers have not been basing their training decisions on concrete and 
comparable information about vendor performance. (MN) 
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ABSTRACT 



The Workforce Investment Act of 1998 requires that, with certain exceptions, 
training services be delivered through the use of Individual Training Accounts (ITAs), 
which participants can use to procure the training of their choice, so long as the training 
program is on a state’s eligible training provider (ETP) list. In March 2000, the U.S. 
Department of Labor made grant awards to thirteen states and local areas as part of the 
ITA/ETP Demonstration, to provide support for ITA and ETP system development and 
encourage innovative approaches and practices. This paper presents interim findings 
from the evaluation of these grantees’ efforts. As such, it describes at about the midpoint 
of the grants’ period of performance what ITA policies and practices were being 
formulated, how ETP lists were being assembled, and what information was available in 
the consumer report systems, a repository of information about eligible programs that 
customers can use to make their training choices. 




EXECUTIVE SUMMARY 



This report has been prepared as part of a contract awarded by the U.S. Department 
of Labor (DOL) to conduct an Evaluation of the Individual Training Account/Eligible 
(ITA/ETP) Training Provider Demonstration. This summary reflects the findings 
reported in the Interim Report for the evaluation; as such, it describes early progress 
made by the demonstration grantees in establishing their ITA/ETP systems, based on site 
visits we made to each of them during the summer and fall of 2000. 

Background 

The Workforce Investment Act (W1A) of 1 998 brought about substantial changes 
in services provided to persons seeking employment and training assistance. An 
important element of W1A is the requirement that training services be provided, with 
certain limited exceptions, through individual training accounts (ITAs), which can be 
thought of as a voucher that customers can use to pay for training of their choice, so long 
as the training program is on an approved list (the eligible training provider list) and 
meets minimum standards of performance. To help customers make prudent training 
choices, information about the eligible programs approved by the state (e.g., costs of the 
training, its duration, and the employment and other outcomes achieved by prior cohorts 
of trainees, among other things) is to be assembled in a consumer report system (CRS) 
maintained by the state and distributed throughout the state’s One-Stop system. The 
establishment of ITAs is intended to empower customers, while promoting accountability 
among states, local areas, and service providers in meeting customers’ needs. 

In the summer of 1 999, DOL issued a Solicitation for Grant Applications (SGA) 
for the Individual Training Account/Eligible Training Provider (ITA/ETP) 

Demonstration. This announcement emphasized that DOL was interested in identifying 
“a national group of vanguard sites” who were committed to implementing ITAs and 
establishing an eligible training provider list that was consistent with WIA and “informed 
by best practice and insight from the field.” Chief goals of the demonstration include 
support for system-building at the state and local levels, rigorous testing of several key 
models or approaches to the establishment of an eligible training provider process and 
ITA payment system, identification of key components of effective ITA implementation, 
and the development of a learning network for information sharing, both across 
demonstration sites and to the larger employment and training system. Each grantee was 
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to receive an amount not in excess of $500,000, for a grant period that was to last 18 
months. 

In March of 2000, DOL announced that it had selected thirteen grantees to 
participate in the demonstration project. Six of these grantees are local workforce 
investment areas (LWIAs) that applied individually or on behalf of neighboring local 
areas. The other seven grantees are states. Of these seven states, four are collaborating 
with some subset of the state’s LWIAs, while the other three states are developing 
statewide systems and strategies. 

The evaluation of the 1TA/ETP Demonstration, being undertaken by Mathematica 
Policy Research (MPR) and Social Policy Research Associates (SPR), consists of a 
process study that entails two rounds of multi-day site visits to each of the thirteen 
grantees. Each site visit entails interviews at both the state and local levels, regardless of 
whether the grantee was itself a state or local area. The Interim Report for the evaluation, 
on which this summary is based, draws on the first round of site visits, which occurred in 
late summer and fall of 2000. Data collection will continue during the summer and fall 
of 2001 , when each grantee will be visited a second time. At that time, we expect that 
service designs and 1TA/ETP systems would have matured substantially. 

Context for the Demonstration 

Site visits to study the 1TA/ETP demonstration grantees were conducted at a time 
when many of them were still developing key aspects of their systems. Moreover, for a 
variety of reasons — including fears of funding shortfalls, a strong economy that makes 
job placements relatively easy to obtain, One-Stop centers’ apprehension of authorizing 
training unless it was absolutely necessary, and the need to have customers go through 
core and intensive services before training could be offered — in some sites no more than 
a dozen customers had been issued an 1TA, out of hundreds of W1A adult and dislocated 
worker enrollees. Clearly, the systems we saw were for the most part very much a work 
in progress. At the same time, all sites had made substantial progress, both in developing 
policies to serve customers with an ITA and in establishing an eligible provider list and 
consumer report system. 

Their progress was facilitated because most were not starting their ITA system 
development from scratch when the ITA/ETP demonstration grants were awarded. In 
fact, almost all had moved sharply away from the exclusive use of contracted training in 
the waning years of JTPA and towards individual referral methods, and over one-half of 
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them claimed previous experience with using vouchers for training, either as a grantee 
under the former Career Management Account demonstration or as part of some other 
pilot program. One-Stop implementation grants that they had received from DOL during 
the mid- to late- 1 990s also helped them establish the infrastructure that they needed to 
serve adult and dislocated worker customers in a W1A framework. 

Building on this framework, the grantees were using their demonstration funding in 
very different ways, which reflected the nature and extent of their prior progress. Their 
grant objectives ranged from the very broad (e.g., develop ITA policies, build a consumer 
report system) to the quite specific (e.g., develop a code of ethics for vendors). In 
general, grantees that specified broader goals were not as far along in ITA/ETP system 
development at the time their grant proposals were prepared. By contrast, those that 
specified narrower goals had many elements of their systems already in place and were 
looking to enhance or refine them in some way. 

Grant objectives can also be categorized with respect to their major area of focus. 
The most common cluster of objectives related to efforts to build electronic consumer 
report systems. Capacity building was another key objective, but the specifics varied. 

One grantee wanted to hire counseling experts to work with participants and coach case 
managers; another wanted to develop a curriculum for a peer-managed workshop; others 
wanted to develop and deliver training workshops for staff or develop computer modules 
that staff could access as a resource. The third largest category of grant objectives related 
to developing or testing ITA policies; for example, one grantee was trying to facilitate 
coordination and joint policy development with adjacent LWlAs. Finally, some grantees 
were using their grant funds to develop fiscal or tracking software or to automate the 
training provider application process. 

Customers’ UseofITAs 

All of the grantees had embraced the ITA model for providing training services and 
generally seemed enthusiastic about its possibilities for empowering customers. In fact, 
nearly all of them were planning on using ITAs for training adults and dislocated workers 
almost exclusively; only two expected to make regular use of contracted training for 
meeting the needs of special populations. However, several expected a sharp drop-off in 
the number of persons they would fund for training each year, citing what they felt was 
WlA’s “work first” emphasis and funding limitations caused by their needing to expend 
resources on developing their core and intensive service strategies. 
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Regardless of the customer volume that they anticipated, sites needed to develop 
policies to guide the way that customers move through core and intensive services, 
because only those who have received at least one service at each of these two service 
levels, without having their employment goals met, are eligible for training. The case- 
study sites varied quite a bit with respect to the policies and procedures that they 
established for this purpose. Some noted that customers whom the case manager felt 
could obviously benefit from training were moved through core and intensive services 
quite quickly. Other sites had more stringent requirements before customers could move 
through to training — for example, by having case managers exhaust all reasonable 
possibilities that the customer might have transferable skills and/or requiring that 
customers spend at least several weeks in core and intensive services engaging in job 
search before training would be considered. 

Another key difference across sites related to how intensive services were used as a 
prelude to training. Some sites felt that they could quickly (in core services) identify 
those who would need training services to meet their employment goals. Such 
individuals were placed in intensive services with the full expectation that they would 
shortly undertake training, so intensive services were focused on helping customers 
develop and refine their career and training plans. Other sites felt that intensive services 
represented another opportunity to identify transferable skills and improve job search 
strategies that might obviate the need for training altogether. 

Despite these different general tendencies from one local area to the next, however, 
all the demonstration sites emphasized that guidelines were not meant to be followed 
rigidly and that they adopted a flexible approach to meeting customers’ needs. It seems, 
then, that a key tenet of WIA that services should be customer driven and based on the 
individual’s own needs appears to have been followed. 

The customer focus is evident as well in the process that sites use to help customers 
make training choices. We identified three models that sites use under various 
circumstances. These are informed choice, which occupies a broad middle ground, and, 
at either extreme, directed choice and free choice. According to the informed choice 
model, One-Stop centers ensure that those authorized for training receive ample 
infonnation, guidance, and assistance, so that they can make prudent choices with respect 
to the occupation for which they want to be trained and the vendor who will provide it. 
This was by far the predominant mode in the sites we visited. Operationally, it meant 
that customers would be required to undertake a comprehensive assessment of their skills 
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and abilities, and engage in labor market and other research, before an ITA would be 
issued. Front-line staff play a key role in serving as “guides” or “facilitators,” striking 
what seemed to be an appropriate balance between lending the benefit of their expertise 
while not being overly directive. 

A key element that made the informed choice approach feasible was that 
assessment and research were required parts of the decision-making process. In addition 
to having participants undertake a comprehensive assessment and engage in labor market 
research, which were everywhere required, some sites required participants to conduct 
field research, such as by visiting several vendors and interviewing former trainees and 
employers who hire in the career area in which the participant wants to undertake 
training. Other sites required that customers attend workshops that are either given by 
case managers or are peer-managed. Sometimes also customers needed to submit a 
formal application, in which they identify the training field and vendor they have chosen 
and justify their decision on the basis of assessment results and the research they have 
conducted. As a consequence of following these steps to having their ITAs approved, 
customers would come to identify appropriate training choices on their own. 

By contrast, a “directed choice” approach was characterized by the case managers’ 
playing a much more directive role. Only one site used this model predominantly, 
although others would use it under special circumstances, as when customers seemed 
unable to make sense of their assessment results or were reluctant to make judgements 
based on the research they had conducted. In these instances, case managers could be 
quite emphatic in steering customers to the choices that the case manager thought best. 

Finally, the third approach, a “free choice” model, was also used sparingly. 
According to this strategy, case managers would essentially give customers free reign to 
make training choices, so long as the training field was for an occupation in demand and 
the vendor appeared on the ETP list. No site used this approach predominantly. 

However, customers who knew exactly what training they wanted to undertake before 
entering the One-Stop center, and who could justify their choice, often had their request 
honored with little difficulty. 

Given the predominance of the informed choice model, our interviews and 
observations lead us to the conclusion that customers are effectively the decision-makers 
almost always. However, their choice is subject to certain limitations established by state 
and local policy. For example, in keeping with the W1A legislation, training can only be 
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funded if it is for an occupation in demand. Some local areas met this requirement by 
drawing on lists developed by the state’s labor market information research unit. Other 
sites used locally- developed lists, and a few had no formal lists but instead relied on the 
judgement of the case managers. Typically, where there were such lists, exceptions could 
be made so long as a prospective trainee could present evidence that a job would be 
available once training was complete; however, a few local areas allowed no exceptions 
whatsoever. 

Other restrictions related to dollar or time limits. Nearly all of the sites set a dollar 
cap on the amount of the ITA that would be funded, but these varied widely across sites, 
from a low of $1,700 to a high of $10,000. Tuition and fees, as well as books, uniform, 
and equipment would normally be funded by the 1TA, and supportive services would be 
provided from a separate pot of money. In keeping with W1A, trainees were typically 
expected to apply for a Pell grant, and amounts they received from that source were often 
applied to the cost of the training, with the ITA paying any balance due. Nearly all sites 
also had time limits on the duration of training that they would support, which they 
usually set at two years. Overwhelmingly, both dollar and time limits were imposed by 
local areas; although they were allowed to impose limits of their own, states generally felt 
that these decisions should be left as a local prerogative. 

Even with these limits, sites could be investing a substantial amount on each 
trainee. For that reason, and also because performance accountability is so central to 
W1A, sites had an interest in doing what they could to ensure that their ITA holders 
completed the training and obtained a well-paying job afterwards. Thus, all sites made 
provisions for keeping abreast of the trainee’s progress and attempted to address 
problems as they arose. Some sites were more proactive than others were, but virtually 
all maintained at least monthly contact with W1A participants in training. 

Given that their performance is publicly displayed as part of the consumer report 
system, vendors also have a clear stake in the trainee’s success, and thus they too played 
a part in monitoring the participant’s progress. Along these lines, proprietary schools — at 
least those that we visited as part of this study — seemed very attentive to students’ needs 
for extra assistance, and were aggressive in helping their students find jobs once the 
training was completed. By contrast, although community colleges offered counseling 
and placement services, they were typically less proactive in their approach. 
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Developing the ETP List and Consumer Report System 

A key element of the training system envisioned by W1A is for there to be clear 
accountability and strong information systems to support customer choice. The eligible 
training provider (ETP) list and consumer report system (CRS) constitute essential tools 
for these purposes. Developing the ETP list and consumer report system proved to be 
extraordinarily resource intensive. 

A key issue that states grappled with as they assembled the CRS was deciding 
whether only ETA- approved vendors should be included or whether it should include non- 
ITA approved vendors as well. About half of the states adopted each approach. Those 
that were developing a restricted consumer report system emphasized the primary 
objective of supporting training customers in selecting a vendor; those that opted for the 
broader approach were giving emphasis to developing a resource for the universal 
customer in core services and making the broadest possible use of the resource they were 
developing. Using different logic, both also saw their approach as serving as an 
inducement for vendors to seek ITA eligibility. 

Regardless of the approach they took, sites were generally eager to widely 
publicize the ETP application process and have as many vendors apply for eligibility as 
possible. In some cases, states took the lead role, such as by sending an ETP application 
packet to all state- certified training vendors in the state, in other states, local areas took 
the lead role, such as by communicating with their former JTPA providers or holding 
informational sessions in the community. Most states attempted to automate the 
application process, both to make it easier for vendors to apply and also to expedite the 
state’s and local areas’ roles in processing applications and entering the data into an 
electronic ETP listing. Those states without an electronic application found the process 
substantially more burdensome. 

Only two states set performance requirements for initial eligibility. The others 
dispensed with such requirements, because they felt ill equipped to make decisions 
regarding performance benchmarks at such an early stage. In general, vendors were not 
even required to submit performance data as part of their initial application. States felt 
that doing otherwise would impose a substantial burden on vendors that they were not yet 
prepared to meet. 

In keeping with the legislation, the approval process for initial eligibility basically 
worked the same way in all the sites we visited — local areas would first review the 
applications, make a judgement of whether the application should be approved, and then 



pass the application on to the state, along with the local area’s recommendation, for final 
disposition. The one difference was that some states had vendors submit their 
applications directly to the local areas, while other states had vendors submit their 
applications to a central state clearinghouse, which then forwarded the applications to the 
local areas for their review. The latter approach was viewed as easier for vendors, who 
would need to submit just one application for each of its programs rather than multiple 
applications to multiple local areas throughout the state; it also standardized the 
application process somewhat. 

Regardless, a concern that local areas expressed was having a vendor’s application 
for eligibility denied by one focal area but approved by an adjacent one. Given that ITA 
holders can hypothetically choose any vendor on the state list, presumably a trainee could 
thus select a vendor that the local area funding the ITA had disapproved. Several of the 
case- study sites were attempting to develop a regional approach to ETP review to 
eliminate this possibility. The case managers in another site admitted that they could not 
envision authorizing an ITA if the customer had selected a vendor that the local area had 
not approved, except under exceptional circumstances. 

Another concern was that many states anticipated a sharp drop in the number of 
vendors who were approved once subsequent eligibility began. As mentioned, only two 
states imposed performance requirements for initial eligibility, deliberately with the 
thought that they wanted as many vendors to apply as possible. Similarly, few reporting 
requirements were imposed, beyond asking the vendors to provide basic information 
about each of their programs, such as the duration of training and its costs. But, for 
subsequent eligibility, vendors must be prepared to begin submitting performance 
information about their programs, relating to the completion rates and employment 
outcomes of trainees, including those that were W1A funded and others. Many vendors, 
especially community colleges, have balked at these requirements. They viewed the time 
and effort necessary to assemble the necessary information as not worth the trouble, 
given that they anticipate serving relatively few W1A- funded trainees. As open- 
enrollment institutions serving diverse community needs, some community colleges also 
feel that their measured performance could misrepresent their actual success, given that 
some of their non- ITA enrollees might lack adequate preparation for training or others 
might not have employment objectives in mind. 

Another reason why the ETP list under subsequent eligibility might be appreciably 
smaller than the initial list is that all states will presumably impose required performance 
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benchmarks for subsequent eligibility, while only two states did so for initial eligibility. 

Few states had made much progress in establishing guidelines for subsequent eligibility. 

All were grappling with where to strike the balance between setting the minimum 
requirements high enough to reflect their high expectations, while not setting them so 
high as to exclude so many vendors that participant choice is seriously compromised. 

Definitional issues were also presenting serious conceptual hurdles. For example, 
even at the point of initial eligibility states needed to make clear what would constitute a 
“program” for purposes of the ETP list. Thereafter, as they attempted to measure 
performance for subsequent eligibility, they needed to decide how key terms would be 
defined, such as who counts as enrolled and what constitutes a completion. Given the 
fact that many community college enrollees may take variable sequences of courses with 
different employment and other objectives in mind, states were generally allowing 
vendors to self-define programs and some were counting as enrollees only those who 
declared their intention to complete the entire sequence of courses that made up the 
program, with employment as the intended objective. 

Amassing the data to measure performance, however the measures were defined, 
also will prove challenging. In keeping with WIA requirements, states were planning on 
relying heavily on using Unemployment Insurance wage records for measuring outcomes 
that were employment related. Thus, vendors would forward the social security numbers 
of enrollees to the relevant state entity, who would then conduct the U1 matching on the 
vendors’ behalf. This approach seemed to make the most sense from the standpoint of 
ensuring completeness, reliability, and comparability. However, the mechanics of this 
process were generally still being worked out. 

In contrast to employment outcomes, where the states will bear most of the burden 
in data collection on the vendors’ behalf, most states are expecting vendors to supply data 
on their programs’ completion rates. One state, however, has agreed to compute even 
these rates for vendors, if the vendors will forward to the state the vendors’ enrollment 
database. 

Because of the complications that needed to be resolved, at the time of our site 
visits only a few states’ systems had any information about vendors’ performance. 

Clearly, much work remains to be accomplished for the consumer report systems to fulfill 
their function of providing an important resource to guide customer choice. 



Vendors’ Reaction 

In the local areas we studied nearly all training vendors certified as eligible for ETP 
purposes are either community colleges or proprietary schools. These two types of 
institutions have very different missions and define themselves very differently. The 
former have traditionally filled an important role in providing training under JTPA, 
because of the breadth of their offerings and generally low tuition, and it is expected that 
they will be similarly important under W1A. However, many are balking at the eligibility 
requirements that W1A imposes, especially the need to submit performance information 
about their programs. In their view, the low volume of ITA-funded trainees that they can 
anticipate does not warrant the time and expense that such a requirement would entail. 
They also fear that their performance would be inaccurately characterized, given the mix 
of customers that many of them serve. 

By contrast, the proprietary schools whose representatives we met characterized 
themselves as being active in the marketplace and highly performance driven. For their 
own purposes, or to meet other state or federal certification requirements, these schools 
had been accustomed to collecting and reporting performance data and saw no difficulty 
with doing so for ETP purposes. They were also highly adaptable and flexible, 
modifying course content, starting times, and training durations to better appeal to 
potential trainees. 

Community-based institutions constitute a third group of potential vendors. While 
we did not visit with any community- based organizations that provide ITA training, 

Local Board and One-Stop staff noted that many of these organizations provided training 
to economically disadvantaged adults under JTPA, but are less likely to be successful 
under an ITA system because of their traditionally narrow customer base. Further, they 
are usually thinly capitalized and are likely to have difficulty coping with an irregular 
flow of ITA students. 

P relim inary Conclusions 

The full ITA and consumer report systems envisioned by W1A were still very much 
under development at the time our site visits occurred. Our upcoming second round of 
site visits, to be conducted in the summer and fall of 2001 , will thus offer the important 
opportunity to view these systems as they have evolved. At this point, however, some 
preliminary conclusions can be drawn. 

1 . State and local-area flexibility seems to be embedded in the systems that are 
developing. Thus, although there are obvious broad similarities in the systems 



that are developing, states and local areas are making unique decisions 
regarding key features of IT A policies. 

2. That flexibility, although clearly embraced by states and local areas as a good 
thing, is causing some confusion and uncertainty at this early stage. Thus, 
some local areas are unsure exactly what their policies and systems should 
look like. In light of this uncertainty, more peer-to-peer exchanges would be 
highly valued, so that sites could share ideas and examples. 

3. Overall, there is likely to be a substantial drop-off in the number of persons 
entering training, at least in WlA’s first full year of implementation. This 
drop-off will come about for a variety of reasons, including a strong economy 
that has made job opportunities plentiful, competing priorities for using scarce 
WLA funds, and case managers’ reluctance to authorize training unless it is 
absolutely necessary. 

4. Sites are maintaining a strong customer focus in the way they approach WlA’s 
three service levels. Despite the reluctance in some cases to authorize training 
unless it is absolutely necessary, in general we observed that sites are highly 
flexible in their approach to customer services. Thus, although sites have 
guidelines for how customers should move through the service levels, it was 
apparent that those guidelines were not meant to be followed rigidly and that 
customers’ obvious needs were taking precedence. 

5. Customer choice, informed by good information, is clearly apparent in the way 
that sites are working with customers to help them select training programs 
and vendors. At the same time, this choice is structured within a framework 
that requires that customers undertake a careful assessment of their skills and 
abilities and conduct extensive labor market and other research. To this 
degree, customers are making choices only after being exposed to a range of 
good information. 

6. Front-line staff are generally playing roles that support informed choice. In 
most local areas that we visited, case managers were playing the role of 
“facilitators,” and were lending the benefit of their expertise without being 
overly directive. To this degree, customer empowerment was being promoted. 
Given that many sites had moved away from contract training in JTPA’s 
waning years, and, in some cases, had previous experience with vouchers, case 
managers often felt that operating under an 1TA system was not that much 
different from what they were accustomed to. 

7. The underdeveloped state of most consumer report systems has meant that 
concrete and comparable information on vendor performance has not been one 
of the sources that most customers have been using in making their training 
decisions. Moreover, given the difficulties that lie ahead, with respect to data 
collection and data management, it will likely be some time before consumer 
report systems are reasonably populated with information about vendors’ 
performance. 
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8. Incentives for community colleges to participate in the ETP system need to be 
established. Proprietary schools as a whole have been very agreeable to the 
requirements that the eligibility process entails. Community colleges, by 
contrast, view these requirements as not worth the effort, given the few ITA 
customers that they anticipate serving, and their focus as educational 
institutions. Given the important role that these institutions play in giving ITA 
holders meaningful choice, sites need to develop strategies to keep community 
colleges as active players in the training marketplace. 

9. The ITA/ETP process is presenting substantial challenges to training vendors 
that relied heavily on workforce development funding under JTPA. Reliant for 
so long on contract training for serving special populations, these 
organizations are finding that their customer flow has been gravely interrupted 
since the enactment of WIA. This challenge has fallen especially heavily on 
community-based organizations. Without an alternative customer base, many 
of these institutions are facing insolvency. Their absence will represent a 
substantial loss to their communities. 

1 0. Processing vendors’ applications and developing the consumer report system 
are extraordinarily difficult and resource- intensive undertakings. In light of 
this, several grantees noted how fortunate they consider themselves to be in 
having been selected to participate in this demonstration, as it has provided 
them with access to special funds for system development that otherwise 
would have needed to come from their regular WIA formula allocation. This 
observation speaks to the difficulty that non-grantee states may be 
encountering with system development, and also for the need for states and 
local areas to develop systems that, once developed, can be sustained at 
minimal cost. 
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I. INTRODUCTION 



The Workforce Investment Act (WIA) of 1 998 brought about substantial changes 
in services provided to persons seeking employment and training assistance. An 
important element of WIA is the requirement that training services be provided, with 
certain limited exceptions, through individual training accounts (ITAs), which provide 
vouchers or related funding vehicles customers can use to pay for training. The 
establishment of ITAs is intended to empower customers to make their own choices, 
while promoting accountability among states, local areas, and service providers in 
meeting customers’ needs. 

To examine how ITAs are currently functioning and how they might be structured 
for maximum benefit, DOL has funded Mathematica Policy Research (MPR) and its 
subcontractor, Social Policy Research Associates (SPR), to conduct an evaluation. The 
evaluation consists of two distinct components. The first is a process study of thirteen 
federally- funded demonstration projects whose progress in establishing ITA and eligible 
training provider (ETP) systems and structures is being examined. The second 
component consists of an experimental evaluation of three distinct models of ITAs that 
vary in the amount of the voucher and the degree to which customer choice is guided by 
the case manager. This Interim Report describes our preliminary findings from the first 
of these two evaluation studies. 

The remainder of this chapter first describes the policy background related to ITAs 
and presents a conceptual framework for the evaluation. Then we summarize the process 
by which DOL selected the demonstration projects to be studied, and we briefly identify 
their salient commonalties and differences. The chapter concludes with a description of 
the research design and methods of data collection associated with our study. 

Individual Training Accounts in the Context of WIA 

The Workforce Investment Act substantially reshaped the nation’s employment and 
training system. One of the primary goals of the legislation is to empower customers to 
take control of their own career and training choices while providing them with the 
information and other supports that they need to choose wisely. One way that local job 
training agencies promote choice is by issuing individual training accounts to adults and 
dislocated workers who are undertaking training. 
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Services Available to Adults and Dislocated Workers 

The W1A legislation requires the use of ITAs only for the adult and dislocated 
worker programs funded under Title 1. As defined by W1A, Section 101, adults include 
all individuals who are 18 years of age or older. Dislocated workers are those who: 

© Have been terminated or laid off (or received a notice of termination or 
layoff) from employment, are eligible for or have exhausted entitlement 
to unemployment compensation (or otherwise show attachment to the 
workforce), and are unlikely to return to their previous industry or 
occupation, or 

© Have been terminated or laid off (or received a notice of termination or 
layoff) from employment as a result of any recent or impending 
permanent plant closure or substantial layoff, or ' 

© Were self-employed but are now unemployed as a result of general 
economic conditions or because of a natural disaster, or 

© Are displaced homemakers, defined to be those who have been providing 
unpaid services to family members in the home, have been dependent on 
the income of another family member but no longer are supported by that 
income, and are unemployed or underemployed and are experiencing 
difficulty in obtaining or upgrading employment. 

The programs for adults and dislocated workers are separately funded, but both 
were authorized to increase the employment, retention, earnings, and occupational skills 
of participants and, as a result, “improve the quality of the workforce, reduce welfare 
dependency, and enhance the productivity and competitiveness of the nation” (W1A 
Section 106). 1 To achieve these goals, adult and dislocated worker programs establish a 
hierarchy of three service levels that consists of core services, intensive services, and 
training services. 

Core services consist of providing basic information primarily intended to assist 
individuals in conducting job search or accessing training or other services on their own. 
In keeping with this, activities authorized by the legislation (Section 134d) as core 
services include: 

® Providing an orientation to services offered under W1A, 



1 Throughout this chapter, and elsewhere in the Report, we identify relevant sections of the WIA 
legislation or implementing regulations as found in the Code of Federal Regulations (CFR), as an aid to the 
reader. 
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® Providing an initial assessment of skill levels, aptitudes, abilities, and 
supportive services needs, 

© Providing job search and placement assistance, and, where appropriate, 
career counseling, 

© Providing information on jobs in demand in the local economy and on 
eligible providers of training services, 

© Describing procedures for how one can access unemployment 
compensation, and 

© Providing follow-up assistance to individuals who were served by the 
program and were placed in unsubsidized employment. 

All adults and dislocated workers are eligible to receive core services. Customers can 
access these services on their own (e.g., through electronic listings of information) or 
with minimal staff assistance. 

Intensive services consist of activities involving somewhat greater staff 
involvement than is typical for core services, but the focus is still on providing 
participants with guidance or information that they need to make informed choices about 
their careers. For example, activities authorized as intensive services can include: 

© Providing comprehensive and specialized assessments of participants’ 
skills or service needs, 

® Developing an individual employment plan for them, 

© Providing group or individual counseling and (for participants undergoing 
training) case management, 

® Delivering short-term prevocational services (e.g., assisting participants 
in developing interviewing skills, communication skills, personal 
maintenance skills, and the like). 

Only persons who have been unable to meet their employment objectives through core 
services will be eligible to participate in intensive services. Moreover, for participants 
served under the adult program, priority is given to public assistance recipients and other 
low- income individuals. 

Training services include skill-building activities that are designed to make 
participants more employable. Training activities that are explicitly authorized (WIA 
Section 134d) include: 

® Occupational skills training, 

® On-the-job training, 
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© Training programs operated by the private sector, 

e Skill upgrading and retraining, 

® Entrepreneurial training, 

© Job readiness training, 

© Adult education and literacy activities (but only if provided in 
conjunction with another training service), 

® Customized training for individuals to be hired by a specific employer. 

Following a hierarchy of service levels, training services are available only to those who 
have undertaken at least one intensive service without it meeting their employment goals. 
Additionally, persons are funded for training only if they are deemed to have the skills 
necessary to succeed in training, select a program of training for jobs that are available, 
and require assistance beyond what is available from other sources (e.g., Pell grants). As 
with intensive services, those funded through the adult program receive priority for 
training if they are public assistance recipients or other low- income individuals. 

With respect to service delivery, core services are to be provided through a One- 
Stop delivery system, which includes at least one comprehensive One-Stop Center in 
each local workforce investment area and potentially a network of affiliated sites. 
Intensive services are also to be made available through the One-Stop delivery system, 
either directly or through contracts with service providers. In contrast, training services 
will generally be delivered by establishing an individual training account that the 
customer can use to procure training services from eligible training providers. Local 
workforce investment areas are required to use ITAs to provide training, except in some 
specified circumstances. As specified by WIA (Section 1 34d4G), these exceptions 
include contracts that the local area can write to fund: 

• On-the-job training or customized training provided by an employer, 

• Training programs by community-based organizations or other private 
organizations that are of demonstrated effectiveness in serving “special 
participant populations that face multiple barriers to employment,” or 

® Other providers, if it is deemed that there are too few providers in the area 
to fulfill the intent of ITAs. 

Role of Individual Training Accounts and Training Providers 

ITAs are intended to transform the delivery of training services to adults and 
dislocated workers by ensuring customer choice. In this regard, once issued an ITA a 
customer will generally be free to use it to procure the training services of his or her 
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choice, subject to some restrictions. First, as noted above, training must be directly 
linked to employment opportunities in the local area (or in another area in which the 
participant is willing to reside). Second, states and local areas may place restrictions on 
the duration of training that can be undertaken and on its costs (20 CFR 663.420). Third, 
customers may use their 1TA only for approved training programs. To attain this 
classification for one or more of its training programs, the training provider must have: 1) 
submitted an application for the program to appear on the eligible training provider list, 

2) agreed to provide information on program costs and on the performance of recent 
trainees for each training program in which the provider wishes to be certified as eligible, 
and 3) meet certain performance benchmarks established by the state and local area. 

With respect to provider eligibility, the state is responsible for developing the 
overall procedures. As outlined in the Act (Section 122), these responsibilities include 
establishing procedures for initial and subsequent eligibility. The procedure for initial 
eligibility is different for postsecondary educational institutions than for other providers. 
Postsecondary educational institutions that award a degree or certificate and are eligible 
to receive funds under the Higher Education Act are to be considered initially eligible, as 
long as they submit an application to the local area containing the information required 
by that area. However, other providers, in their application for initial eligibility, may be 
asked to submit program- specific information about the costs of their training services 
and the outcomes achieved by former trainees, and have met performance levels if 
required to do so by the state. 2 The state is to establish these requirements after a 
consultation process involving Local Boards, business, and labor. 

Initial eligibility can last from 12 to 18 months. Thereafter, all providers who seek 
certification, including postsecondary degree-granting institutions, must submit 
performance and cost information and meet performance criteria annually, in order to 
maintain subsequent eligibility (W1A Section 122c and d). This performance information 
should cover: 

® Outcomes achieved by all individuals who participated in the programs 
for which the provider is seeking certification, including: 

' - The percentage of those participating who completed the training. 

- The percentage of those participating who obtained unsubsidized 
employment. 



2 New training programs are, however, exempted from meeting the performance criteria. 
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- The wages at placement of all individuals who participated. 

° Outcomes achieved by individuals funded by WIA who participated in the 

programs for which the provider is seeking certification, including: 

- The percentage of those participating who completed the training 
and obtained unsubsidized employment. 

- The percentage of those who completed the training who were 
found to be in unsubsidized employment six months after being 
placed in employment. 

- Wages received by those who completed the training, measured 
six months after the first day of the employment involved. 

- If applicable, the percentage of graduates who obtained a license 
or certificate, an academic degree or equivalent, or other measures 
of skills. 

The required performance levels on these measures are established by the states, although 
local areas can establish higher levels. In any case, the levels established for a provider 
should take into account characteristics of the local economy and of the persons it serves. 
Performance information on the employment measures must be compiled using data from 
the Unemployment Insurance (UI) system. 

According to WIA, the local boards should review all applications, whether for 
initial or subsequent eligibility, and then forward the list of providers they have approved 
to the state. The state, in turn, is to check the performance data for accuracy and compile 
a master state eligible training provider list, which is disseminated to all local areas along 
with the cost and performance data. This complete information packet is referred to as 
the “consumer report system,” and should include any additional information “necessary 
for an adult or dislocated worker customer to fully understand the options available to 
him or her in choosing a program of training services” (20 CFR 633.570). 

Conceptual m odel for the Evaluation 

The use of ITAs represents a dramatic shift in the way in which training services 
are delivered, with potentially important implications for the ways in which One-Stop 
Centers, vendors, and customers relate to each other. However, although the broad 
guidelines described above are prescribed by the WIA legislation and implementing 
regulations, states and local workforce investment boards have substantial discretion in 
how ITAs are developed and used in practice, for example with regard to the 
performance levels required of eligible providers, the information included in consumer 
report systems, the types of customers for whom training services are deemed 
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appropriate, the guidance customers are given to assist them in making training choices, 
the ways in which vendors are monitored, and the types and duration of training that is 
considered “fundable.” To help us understand this variability, and how it impacts the 
delivery of training services, we have developed two heuristic models. The first is a 
system- level model of ITA implementation, depicting how the system of training services 
might be expected to develop. The second model depicts a customer- level view of how 
clients might flow through the system, and it identifies quality indicators for the various 
steps of the process. 

System-Level Model of ITA Implementation 

ITA implementation represents a wholesale transformation of how the various 
actors in the training system — including the federal and state governments, local 
workforce boards, and vendors — relate to one another. An important objective of this 
evaluation is to describe these changes. To clarify these interrelationships, we outline 
below the roles the various actors are expected to play. 

The Federal Role. The WIA legislation itself and the implementing regulations 
developed by DOL constitute the broad guidelines within which the ITA system will 
develop. Because the legislation and regulations will not vary, at least over the short- 
term, they constitute a context for the development of the ITA system rather than an 
important source of site-to-site variation. Nonetheless, the guidance that DOL issues and 
the training and technical assistance it provides can be expected to evolve over time, 
partly in response to input from states and local areas as they grapple with the 
practicalities of implementation. 

State Decisions and Influences. Under the WIA legislation, Governors play an 
important role in setting guidelines for their local workforce investment areas’ use of 
fTAs. Their responsibilities include establishing criteria for determining the eligibility of 
training providers, both initially and subsequently. As part of the certification process, 
states also develop monitoring and enforcement procedures to ensure that providers 
report information accurately and comparably. Similarly, procedures for sanctioning 
providers who report inaccurately or whose performance slips below specified thresholds 
must also be developed. Finally, with respect to certification, the state designation board 
maintains and updates the state- approved list of eligible training providers and 
disseminates the list along with each provider’s performance and cost information. States 
may vary in the procedures they establish for these purposes, including how local boards 
(and potential trainees) access the list, how the list is updated, and the ways in which 
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performance and cost information are conveyed to allow meaningful comparisons across 
programs of different types. Some states might also include on a master list providers 
who are not deemed eligible under W1A Title I; the advantages of doing so would be to 
make available a comprehensive information tool to be used by customers in core 
services or by partner programs. 

States can also set limits on the types or duration of training that will be funded (20 
CFR 633.420). States may establish these restrictions as upper limits on either the length 
of training that will be supported, the costs incurred, or both. Or they may establish 
limits for individual customers based on their training needs (e.g., depending on which 
field of study they are undertaking). Similarly, they can restrict the period of time over 
which trainees can access funds in their training accounts. An important objective of the 
evaluation is to understand how and why these decisions were made and to track their 
implications for the design and delivery of training services at the local level and from 
the standpoint of ensuring customer choice. 

An additional state role lies in training and capacity building. The WIA legislation 
(Section 134a) gives states discretion to use their WIA funds for these purposes, 
including providing assistance and training to One-Stop operators, One-Stop partners, 
and eligible providers. The type of assistance that is provided can clearly be expected to 
influence the service delivery process. 

By way of establishing a context, it is also important to note the prior progress that 
states have made in implementing many of the tenets of WIA. Although ITAs represent a 
relatively new development in the delivery of workforce training, many states had 
already made substantial progress towards implementing a One-Stop service delivery 
system during the last few years of the Job Training Partnership Act (JTPA), which WIA 
replaces. As part of this process, many developed highly detailed data banks with 
sophisticated user interfaces to help customers make career choices and access 
information about training providers. 3 Thus, as the demonstration begins, states find 
themselves at uneven stages in the development of the infrastructure that WIA mandates. 
Their recent experience with One-Stop implementation, which was in progress at the time 



3 DOL awarded One-Stop implementation grants to states and local areas in the several years before 
WIA was enacted. The first round of implementation grants to nine states was awarded by PY 95. 
Additional states received grants in waves from subsequent rounds of funding through PY 98. 
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DOL awarded the demonstration grants, and the data systems that had already been 
constructed are important for understanding subsequent developments. 

Influences and Responses of the Local Areas. Local boards are responsible for 
implementing an ITA/ETP system that meets state and federal guidelines, while being 
responsive to the needs of their customers. As part of their responsibilities, these boards 
must identify providers who are designated as eligible vendors according to the criteria 
established by the Governor, and they can impose additional performance criteria. In 
addition, they may impose limits on the types of training that will be funded (e.g., to 
ensure that the proposed training is directly linked to employment opportunities in the 
area) and the dollar amount of training that will be funded for different types of 
customers. 

Payment mechanisms that local areas establish might also vary, including how and 
when payments are made to providers. Evidence from the Career Management Account 
(CMA) Demonstration, which tested ITA-like procedures, suggests that different funding 
mechanisms can be used, including credit card- like transactions, a checkbook or 
payment -coupon system, and trainee identification cards to establish eligibility. 4 
Similarly, payments can be tied to benchmarks relating to the trainees’ performance (e.g., 
satisfactory attendance or progress must be maintained). 

Local Boards also have discretion in determining whether and under what 
circumstances exceptions to the use of ITAs will be considered. Such instances might 
include when there are an insufficient number of eligible providers in the area or a CBO 
of demonstrated effectiveness in serving special populations is identified. Conversely, a 
Local Board may decide to extend the use of ITAs to customers beyond those served with 
WIA Title 1 funds, such as for persons undergoing training funded by a partner program. 

Staff members of One-Stop Centers also have substantial discretion in determining 
who is eligible for training services. Given stipulations in the legislation, decisions must 
be made regarding who is in need of and able to successfully complete training, and in 
what manner priority for public assistance and other low- income individuals will be 
established from among those served in the adult program. Similarly, because only those 
who have received core and intensive services without having their needs met are eligible 



4 The CMA Demonstration was launched in the summer of 1995 when DOL awarded grants to 13 
local areas. 
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for training services, we need to understand how services are tiered within the One-Stop 
Center and how customers move through the various service levels. Finally, within the 
framework of providing customers with the information they need to make informed 
decisions, counselors perform the important role of helping customers interpret 
assessment results, assisting them in developing an individual employment plan, and 
weighing the advantages of choosing from among alternative providers. Results from the 
CMA Demonstration suggest that some case managers have difficulty providing helpful 
guidance along these lines while relinquishing control over the ultimate decision-making. 

Local areas’ prior experience with voucher- like systems is also important to this 
evaluation. For example, some sites selected for the evaluation have participated in the 
CMA Demonstration. Following many of the principles of ITAs, this demonstration was 
similarly designed to increase customer choice while allowing individuals to control their 
career and training plan. Obviously, lessons local areas have learned from this prior 
experience will influence developments under the current demonstration. More broadly, 
many local areas had already moved away from class-size training and towards individual 
referrals in the last several years of JTPA. 

Finally, as part of the efforts that they have already undertaken to promote self- 
services within a One-Stop context, some local workforce investment areas have begun 
assembling information about training providers, including data concerning provider 
performance. Other local areas, by contrast, are just beginning the process. 

Vendors’ Influences and Responses. Under the ITA system, vendors’ roles and 
relationships to other actors have changed dramatically in comparison to the situation that 
was typical under JTPA. Under WIA, vendors who are interested in being certified as 
eligible have a primary obligation to demonstrate their effectiveness to the state and local 
boards, as evidenced by their program completion rates, employment rates, wages at 
placement, or other information required by the eligible provider system. Accordingly, if 
they have not already done so, vendors must develop and maintain the data systems 
necessary to track their performance. 

Vendors may also have an interest in marketing their services directly to 
prospective trainees, who now have much more discretion than previously in choosing 
the training venue that they feel is most appropriate. Vendors may also respond by 
modifying course offerings, training duration, and fee structures to make their offerings 
more appealing to consumers. 
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Customer-Level Model of Training 

In addition to understanding the policies and procedures that states and local areas 
develop, an objective of this evaluation is to examine how customers access training and 
the types of choices they make. To provide a framework for evaluating this process, we 
draw on indicators of high-quality service planning that we have developed and applied 
in many of our previous evaluations for DOL. 5 

Levels of Service. To understand fully the assistance that trainees receive, we need 
to examine what core and intensive services have been provided before the customer 
accessed training services. According to W1A, only those who have been provided at 
least one core service and one intensive service are eligible to receive training services. 
Local areas might have very different policies regarding what core and intensive services 
are provided and for what duration, before access to training services is permitted. For 
example, in some cases, one core and one intensive service might be provided quickly — 
even in a single visit to the One-Stop Center, when it is clear that a customer will need 
training services to be successful in the labor market. In other local areas, a more 
protracted service sequence may be required. Thus, customers may have had very 
different service histories before an ITA is issued. 

Assessment and Service Planning. Customers undertaking training will normally 
have an individual employment plan (IEP). As defined by the WIA regulations, the IEP 
should be jointly developed by the participant and the case manager and identify “the 
participant’s employment goals, the appropriate achievement objectives, and the 
appropriate combination of services for the participant to achieve the employment goals” 
(WIA Section 134d3C). 

Under WIA, customers have a substantial new responsibility in developing the IEP 
and carrying it out. Case managers, in turn, have an obligation to provide potential 
trainees with the guidance and information to enable them to make informed choices. As 
part of a high-quality service-planning process, customers should be provided with a 
comprehensive assessment of their basic and occupational skills, interests, and aptitudes, 
leading to a determination of what skills the customer already possesses and which 
additional ones are needed to ensure competitive performance in the labor market. The 
assessment should lead to a determination about whether or not additional training is 



5 See, for example, A Guide to Well-Developed Services for Dislocated Workers, ETA Research and 
Evaluation Report Series, 1994. 
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desirable and, if so, for what type of training the customer is best suited. At the same 
time, supportive service needs should be evaluated, so that potential barriers to a 
customer’s successful completion of training can be redressed. 

Customers consult with their case managers using the assessment results as a tool 
in developing the 1EP. As a first step, case managers can provide career counseling and 
assist the customer in career exploration. A high-quality 1EP should then clearly identify 
the customer’s short- and longer-term employment goals and the training objectives for 
achieving those goals, including interim benchmarks. The plan should be tailored to the 
skills, interests, and background of each participant, and should involve the customer in 
the service planning process. Finally, the IEP should establish high yet realistic 
expectations for learning and employment success, and should establish clear and 
appropriate goals that the participant is committed to achieving. With respect to training, 
case managers must balance the need to guide the participant in his or her decision- 
making while allowing for maximum customer choice; how this balance is struck may 
vary across local areas, and, within each area, from case manager to case manager. 

Linking with Other Funding Sources. According to the WIA legislation (Section 
134d4B), program operators are expected to coordinate training funds that might be 
available from other sources, including Pell grants. To ensure that WIA funds are used 
most effectively, case managers should therefore explore these alternative funding 
sources with participants and assist them in accessing these sources whenever feasible. 
Similarly, WIA funds are to be used for meeting supportive service needs only when 
participants are unable to have their needs met through other sources. These strictures 
thus impose the responsibility on program operators to work collaboratively with other 
funding agencies and to leverage other resources when feasible. Thus, it is important to 
document what non-ITA resources customers have access to and from what sources they 
derive. 

Training Content, Duration, and Quality. As part of the evaluation, we also 
need to understand what types of training customers actually undertake. One reason that 
training choices will vary across sites is because the extent of empowerment engendered 
by ITAs will also vary. For example, in some local areas ITAs may present customers 
with a bewildering array of choices, while in places with few vendors customer choice 
will be quite limited. Even in areas with extensive choice, customers must negotiate with 
their counselors to complete a training plan that adheres to state and local policy 
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guidelines (e.g., based on what occupations are determined to be in demand, with respect 
to limits on costs or duration of training, etc.). 

We also need to document how customers make choices, based on what 
information is provided to them. This information can include assessment results, 
discussed above, but also the consumer report system, which is designed to provide 
customers with rich information about vendor performance and costs, along with 
additional data that may help them make a training choice. Customers may also respond 
to marketing appeals made by vendors, or have strong prior preferences about what 
training they want to undertake. Accordingly, some individuals may be little influenced 
by either vendor performance, assessment results, or the advice and guidance provided by 
a counselor, and instead rely on more “subjective factors” in making their decisions. 

How these diverse factors interplay in leading to training choices and how they vary 
across local areas are key areas of inquiry for this evaluation. 

Grantees Selected for the Demonstration 

In the summer of 1999, DOL issued a Solicitation for Grant Applications (SGA) 
for the Individual Training Account/Eligible Training Provider (ITA/ETP) Demonstration 
(SGA/DFA 99-01 7). This announcement emphasized that DOL was interested in 
identifying “a national group of vanguard sites” who were committed to implementing 
ITAs and establishing an eligible provider list that was consistent with WIA and 
“informed by best practice and insight from the field.” Chief goals of the demonstration 
include: 

® Support for system-building at the state and local levels. 

® Rigorous testing of several key models or approaches to the establishment 
of an eligible training provider process and ITA payment system. 

® Identification of key components of effective ITA implementation. 

® Support for demonstration “learning laboratory” sites in designing and 
implementing innovative processes and systems. 

® Development of a learning network for information sharing, both across 
demonstration sites and to the larger employment and training system. 

Each grantee was to receive an amount not in excess of $500,000, for a grant period 
that was to last 18 months. In support of the demonstration’s goals, the SGA. stipulated 
that funds could be used for a variety of activities, including: 
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© Designing and delivering in-service training courses that would help case 
managers understand their new role in supporting customer decision- 
making. 

® Developing orientation sessions to help customers understand their 
opportunities and responsibilities. 

© Developing orientation sessions on ITAs and the eligible provider process 
for vendors. 

® Developing a system to track training provider performance. 

© Developing an 1TA expenditure reporting system to provide individuals 
with information on the status of their account balances. 

© Supporting related efforts associated with implementing ITAs and the 
eligible training provider system, including travel for staff. 

The SGA emphasized that funds were not intended to be used for direct training 
expenditures of customers, but rather for “building the ITA/Eligible Provider system.” 
However, grantees were allowed to use up to 20% of their grant award to fund such 
training, to supplement other funds that would also be used for this purpose. 

To be consistent with the intent of the WIA legislation, DOL noted that plans for 
1TA systems being proposed by bidders would need to adhere to a number of guidelines 
on which substantive variability could not be allowed. Thus, all grantees would be 
required to include all adult 1TA participants in the demonstration, inform participants of 
the dollar amount available in their 1TA accounts, make available the performance and 
cost information on vendors that WIA requires, pay vendors on a timely basis, develop an 
IEP for each individual undertaking training, and inform staff of the role they are 
expected to play in the ITA process. Additionally, bidders were informed that they 
would be expected to administer a common customer satisfaction survey to both all 
participants and all staff. 

There was nonetheless quite a lot of room for variability in the systems that 
grantees could establish, including with respect to these key features: 

© How the dollar amount of the ITA is determined (e.g., fixed for all 
participants or varying by assessment results). 

© Who approves the ITA (e.g., a counselor, a committee, etc.). 

® What career direction is allowed and how informed choice is provided 
(e.g., how much leeway are customers given in choosing a career 
direction, how are demand occupations identified, what is the role of the 
case manager). 



° Who has final authority on selection of the training provider. 

© What the 1TA can pay for. 

Applicants responded to these guidelines and, on March 27, 2000, DOL announced 
that it had selected thirteen grantees to participate in the demonstration project. These are 
identified in Exhibit 1- 1 . As the exhibit shows, the grantees differ with respect to the 
nature of the partnership that applied for the grant. Thus, six grantees are local workforce 
investment areas (LWlAs) that applied individually or on behalf of neighboring local 
areas. The other seven grantees are states. Of these seven, four are collaborating with 
some subset of the state’s LWlAs, while the other three states are developing statewide 
systems and strategies. 

Research Design and Data Collection 

To investigate the issues described earlier in this chapter, the evaluation of the 
1TA/ETP Demonstration consists of a process study that entails two rounds of multi-day 
site visits to each of the thirteen grantees. In general, each site visit will entail interviews 
at both the state and local levels, regardless of whether the grantee was itself a state or 
local area. We believe that this approach is sensible given the inevitable interplay 
between state and local decisions, in keeping with legislative requirements. Nonetheless, 
because the grantees are so different with respect to whether the primary activity 
associated with the grant is at the state or local level, we will “customize” the site visit 
agenda to spend more of the time on site at the grantee’s own level (i.e., by spending more 
time at the state level and less time at the local level for state grantees, and vice versa for 
grantees that are local areas). 

The first round of site visits, which occurred in late summer and fall of 2000, forms 
the cornerstone for this report. Overall, the objectives of these visits were to understand 
states’ and local areas' efforts at implementing the ITA and ETP systems, impediments to 
implementation that were encountered, and variations in key elements of the emerging 
systems. In investigating these issues, we engaged in approximately three days of data 
collection associated with each grantee that entailed: 

® A review of written guidelines and plans regarding the use of ITAs. 

© Discussions with the demonstration grant administrator. 

© Discussions at the state level with administrators responsible for the 
1TA/ETP system, to learn about the process and procedures that states 
have established. 
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Exhibit 1-1 

ITA/ETP Grantees and Local Areas Selected for the First Round of Data Collection 
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© Discussions with state and local MIS staff, to leam about the computer 
interface for the consumer report system, tracking, and the like. 

© Discussions with local WIB staff and administrators of the One-Stop 
Center, to leam how the local area has provided input into the state 
guidelines regarding ITAs and what additional guidelines the area has 
established on its own, and how these guidelines have been implemented. 

© Discussions with case managers and counselors, to understand the client 
flow-through process, including what types of assistance clients are given 
in making their training choices. 

© Meetings with approximately two vendors, to discuss the ways in which 
they have responded to the WIA requirements. 

Because all first-round site visits entailed data collection at both the state and local 
levels, a key decision involved deciding which local area or areas should be visited in 
conjunction with the study of state grantees. These areas were selected in consultation 
with the state ITA/ETP grant administrator and generally consisted of local areas that had 
some responsibility for carrying out the state’s grant activities. Exhibit I- 1, which was 
previously referred to, identifies these choices. 

Although this Report is based on the first round of site visits, data collection will 
continue during the summer and fall of 2001, when each grantee will be visited a second 
time. Because the first round of site visits will have established the context in which 
grantees are operating, the second round of site visits will focus on the evolution of the 
ITA and ETP systems and on the actual process of service delivery at the local level. 

Thus, during the return visits more attention will be focused on the local level, even for 
grantees that are states, so that we can better understand the process by which clients flow 
through the system, their reaction to the ITA system, and the characteristics and quality 
of the services that result. In general, for state- level grantees we will visit local areas 
during the second round of data collection that are different from those we visited during 
the first round. 

This Interim Report was prepared for DOL and is intended as a comprehensive 
account of the first round of data collection. It includes, in Chapter II, a discussion of the 
context for implementation, by reviewing the grantees’ grant plans and their prior 
experience with vouchers and progress towards WIA implementation. Chapter III 
provides a discussion of ITA policies and procedures, including the priority given to 
training services, limits on the use of ITAs, and the client flow-through process with a 
focus on how customers become eligible to access an ITA and how they make their 
training choices. Chapter IV discusses the infrastructure supporting the ITA system, 



including how vendors became eligible to provide training services and how vendor 
performance data are being collected and displayed to customers. The vendors’ response 
to the 1TA/ETP system then becomes the focus for Chapter V. Finally, Chapter VI 
concludes the Report with some summary observations. The Final Report, to be 
completed early in the year 2002, will provide a detailed accounting of the study’s 
findings, using information gathered from both rounds of data collection. 



II. GRANT PLANS AND IMPLEMENTATION PROGRESS 



As discussed in the previous chapter, our evaluation focuses on the thirteen 
grantees selected to participate in the 1TA/ETP demonstration. At the time these grants 
were awarded, the grantees differed with respect to their prior experience in using 
vouchers and, in general, with their readiness to operate under WIA guidelines. Partly 
for this reason, they articulated different objectives as part of their grant plans and 
experienced varied success in carrying out their grant objectives. To set the stage for the 
report’s subsequent chapters, we now describe these differences. 

The Grantees’ Prior Experience and Readiness 

At least in principle, the use of ITAs, along with the concomitant obligation for 
states to identify eligible providers and develop a consumer report system, could 
represent a substantial transformation in the way training services are delivered. Along 
these lines, and in keeping with the principles of WIA, customers can become 
empowered to take control of their own training choices. Conversely, case managers 
need to adopt a coaching approach in working with their customers, rather than making 
decisions for them. The use of ITAs consistent with WIA also necessitates substantial 
effort towards building and managing information systems, such as in processing 
vendors’ applications for eligibility, amassing information required for the consumer 
report system, developing searchable databases for customers to use in identifying and 
comparing eligible training providers, and building systems to track the commitment and 
expenditures of vouchers. These tasks are all potentially enormously challenging. 

The extent to which they were so in fact, however, was contingent on both the 
grantees’ prior experience with using voucher- like systems for training and their prior 
state of WIA readiness. 

Prior Experience with Using Vouchers 

A key provision r of the system of training for adults and dislocated workers 
mandated by WIA enables prospective trainees to select a program of study and a training 
vendor that best meet their individual needs. This method for delivering training services 
stands in contrast to arrangements common under JTPA, whereby job training agencies 
contracted with training providers to deliver class-size training customized to a cohort of 
JTPA trainees. Although contracted training is still allowable under WIA, this legislation 
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makes clear that such arrangements should be developed only under certain limited 
circumstances. 

However, in actuality most of the 1TA/ETP grantees had gradually moved away 
from the widespread use of contracted class-size training during JTPA’s waning years. In 
its place, customers were typically enrolled in occupational skills classroom training on 
an individual-referral basis. Indeed, one site claimed to have predominantly used 
individual referrals for training for decades and that over 95% of its JTPA training was 
conducted this way. 

Even more importantly, prior to the onset of the ITA/ETP Demonstration at least 
eight sites had explicitly used vouchers for arranging at least some training. The most 
common vehicle for doing so was as part of the Career Management Account (CMA) 
Demonstration, described in the previous chapter. This demonstration project was 
designed to test the feasibility of providing training for dislocated workers through 
vouchers, so that customers would have maximum flexibility in selecting programs of 
study and training sources. A half-dozen ITA/ETP grantees (or their local-area partner) 
participated in CMA, either as a direct grantee or as a member of a consortium; these 
include Metro Portland, Baltimore, the Atlanta Regional Commission, Missouri (SDA II), 
Texas (East Texas), and Ohio (Cincinnati). 

Some ITA/ETP grantees also noted that they had even earlier experience with using 
vouchers. In some cases, this experience is long-standing and far-reaching. For example, 
the Atlanta Regional Commission first began using vouchers as a way of providing 
training services to the approximately 13,000 workers who were dislocated in 1991 when 
Eastern Airlines went bankrupt. Given its existing infrastructure and the size of the 
dislocation, this local area was facing many more prospective trainees than it could 
readily serve with class-size training. As a result, it used a voucher system and largely let 
dislocated workers choose whatever training they wanted. Similarly, Metro Portland has 
a strong history of using vouchers. In addition to its participation in the CMA 
demonstration (as part of a consortium), this local area has embraced what it calls 
Individual Learning Accounts in a major way over the last several years. Like ITAs, 
Learning Accounts represent efforts to put money in the hands of social service 




11-2 



37 



customers, who can then use these funds to advance their careers through continuing 
training and education. 1 

The prior experience that many of the grantees have had with voucher systems and 
individual referrals for training left them much better prepared for adopting ITAs as the 
mechanism for providing training under W1A than would otherwise have been the case. 
For example, respondents at several of our case-study sites remarked that case managers 
had moved years ago to adopt a coaching approach to helping customers make training 
choices. As a result, they felt that, although the transition was difficult at the time, ITAs 
meant little change to the way they have been doing business over the previous several 
years. For example, Baltimore moved to implement a voucher system when it became a 
CMA demonstration grantee in 1995 and has stuck with this approach ever since. As part 
of the CMA experience, the workforce area facilitated a change in the way case managers 
performed their job, from being “paper-pushers” under JTPA, to becoming empowered to 
make decisions regarding how to serve clients as individuals with unique needs. Some 
case managers found this transition difficult, and some are still adapting to their new role, 
but the bulk of the turmoil associated with the readjustment occurred before W1A 
implementation. 

Additionally, information systems of the kind envisioned by WIA’s call for a 
consumer report system were already under development in some cases. For example, 
the Atlanta Regional Commission found that many of the dislocated workers whom it 
served through vouchers as part of the Eastern Airlines bankruptcy had made poor 
training choices (e.g., cosmetology, bartending, and other jobs with poor career 
prospects). Consequently, it began to systematically build a vendor list and carefully 
monitor vendor performance, so that it could provide vendors’ performance information 
directly to participants. Subsequently, by participating in the CMA demonstration it was 
able to further develop and systemize its procedures. Similarly, Pittsburgh shifted from 
class-size training to what the city’s WIA manager termed a voucher system in 1996. As 
part of this process, the local Private Industry Council developed a performance review 
system for training vendors, known as the Singh report. The report was widely accepted 



1 Typically, Learning Account customers make regular contributions to their account, and these are 
matched by contributions made by employers and social service agencies. Funds can then be used to pay 
for education and training expenses. 
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by program administrators as a powerful tool to aid customers in identifying effective and 
ineffective programs. 

Although grantees that had prior experience with vouchers had a head start towards 
1TA implementation, the transition has not been entirely trouble-free. One grantee, for 
example, noted that serving exclusively dislocated workers with vouchers, as it did under 
CM A, turns out to be a very different experience than serving low- income or welfare 
adults with ITAs, as it is doing under W1A. Customers in the former group, this grantee 
feels, are much more self-sufficient and need much less assistance and support than those 
in the latter group. Accordingly, the “light-touch” services that it had in place to move 
dislocated workers through the service levels, from self-directed job search to assessment 
and then referral for training, needed to be substantially intensified to meet the needs of 
low- income adults. Other grantees noted that, although case managers may had already 
come to adopt a coaching role to helping customers make training choices, W1A 
represented a new environment in which to do so. Under JTPA, for example, customers 
and case managers shared the presumption that all customers who came in for services 
would undergo training. By contrast, W1A, with its stipulations that training should be 
considered only when core and intensive services were unable to yield satisfactory 
results, changes this mindset completely. Thus, some case managers felt that under WIA 
they were taking much more time with each client, and thereby came to understand the 
client’s training needs and pre-existing skills and abilities much more completely. On the 
other hand, they often needed to overcome customers’ mistaken assumption that WIA, 
like JTPA, was “a training program” and that all customers would automatically be 
approved for training. 

More generally, even grantees that had prior experience with vouchers, such as 
Baltimore, noted that staff training and capacity building needed to be ongoing efforts if 
the intent of WIA to empower customers was to be fully realized. Moreover, nearly all 
grantees found themselves far from having in place the informational infrastructure that 
the consumer report system requires, as a subsequent chapter will describe. 

Overall WIA Readiness 

Providing training through ITAs in a manner that is consistent with WIA requires 
more than developing 1TA and ETP processes and procedures. On the contrary, ITAs 
need to be delivered as part of a comprehensive system for delivering employment and 
training services to a broad spectrum of adults and dislocated workers. This requirement 
entails establishing a One-Stop service delivery system, including the articulation of well- 
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developed self-service and staff-assisted core services and intensive services. In other 
words, in order for the 1TA/ETP grantees to be effective in carrying out their grant 
obligations, they of necessity need to be operating within the context of a much more 
complex network of partners and services. 

The level of WIA readiness varied substantially in the states and local areas that we 
visited, and this variation influenced to some degree how successful the grantees were in 
establishing their ITA/ETP systems by the time of our first site visits. Some grantees 
benefited greatly because the local area we studied or the state of which it was a part had 
long anticipated some of the changes required by WIA and had moved to develop a One- 
Stop delivery system well before WIA was enacted. For example, One-Stop Centers had 
been long established and were functioning quite smoothly in Texas quite a few years 
before our site visit occurred. Their development came about as a result of the Texas 
Workforce and Economic Competitiveness Act of 1993, which mandated the 
establishment of Career Centers that were to include the participation of JTPA, 
Employment Services, Unemployment Insurance, and various education programs. 
According to the legislation, each center was to provide labor market information, 
common intake and eligibility determination for all local workforce development 
programs and services, the independent assessment of individuals’ needs and the 
development of individual service strategies, coordinated case management and 
counseling services, supportive services, and the use of individual referral for basic 
education and classroom occupational skills training. As a result of this head start, by the 
time the grant solicitation was announced, Texas had established over one hundred local 
Career Centers, of which 50 were characterized as WIA full- service centers. 

In at least this one case, One-Stop service systems were to some degree home- 
grown and sprung from strong state- level initiatives. In our other case-study sites, 
however, One-Stop system building generally only began when the state was awarded a 
One-Stop implementation grant expressly for this purpose. These grants — which were 
awarded by DOL to all 50 states over a period of years from the mid- to late- 1 990s — 
provided much needed financial resources that states and local areas used to modify 
infrastructure, purchase new equipment, and develop and support self-service resources 
and tools. Similarly, the national network of support that developed around One-Stop 
system-building provided a fertile ground for the incubation of new ideas and the 
dissemination of information about promising approaches and best practices. Indeed, 
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across all the states that we studied, the One-Stop implementation grants, and the DOL 
vision that they represented, constituted a critical formative influence. 

That uniform advantage notwithstanding, some states were clearly further along in 
the entire process of W1A implementation than others. A few of the states covered by the 
1TA/ETP demonstration (i.e., Texas, Indiana, and Pennsylvania) had applied to be WIA 
“early implementers,” which means that they were attempting to make the transition to 
WIA before the required start date of July 1 , 2000. Some others had not applied for early 
implementation status, but nonetheless were also far along in their WIA transition 
process. As such, they had their state and local boards in place, had One-Stop systems in 
place, and, in general, had established necessary and relevant workforce policies and 
procedures. These states and the local areas that they represented were able to move 
quickly to establish their ITA/ETP systems, as was apparent when we first visited them in 
the summer and fall of the year 2000. 

By contrast, other states were less ready with respect to the essential elements of 
WIA, and it typically slowed their ITA/ETP system development substantially. For 
example, one state was going through a somewhat difficult reorganization of its local 
areas at about the time the grant solicitation was announced. The controversy and 
disruption that this caused delayed One-Stop system building in the state such that no 
full-service centers had yet been chartered in the local area that we were going to visit as 
of the late summer of 2000, and virtually no one had yet been issued an ITA. 

In another example, one local area itself was substantially far along in making the 
transition to WIA at the time the ITA/ETP grants were awarded and in fact had been 
designated as an early implementer under WIA. By contrast, during this same time, the 
state was undergoing some major revisions to its WIA State Plan, including a 
reconsideration of the definitions of the three service levels and the sequence of services, 
a development that was hampering the local area’s efforts to finalize its ITA policies and 
procedures. 

Grant Plans and Grant Implementation 

All of the ITA/ETP grantees are ultimately striving to obtain the same objectives — 
well-developed procedures for issuing IT As, a comprehensive and readily accessible 
consumer report system, and a mechanism for tracking the obligation and expenditures 
implied by issuing vouchers. That said, some grantees were further along than others in 
establishing elements of these systems by the time the grants were issued, partly as a 
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function of their prior experience with vouchers and overall state of W1A readiness. 
Largely for this reason, they have identified different specific objectives in their grant 
plans and are consequently planning on using their grant funds in divergent ways. We 
identify some of these differences in Exhibit 11-1, along with an indication of how 
progress towards grant implementation has proceeded to date. 

As a glance at the table suggests, grantees have identified quite a wide variety of 
grant objectives. These range from the very broad, such as to further the development of 
1TA policies and ETP systems (several states), to the very specific, such as to develop the 
NC STARS system (North Carolina) or develop a code of ethics for vendors 
(Pennsylvania). In general, grantees that specified broader goals were not as far along in 
1TA/ETP system development at the time their grant proposals were prepared. By 
contrast, those that specified narrower goals had many elements of their systems already 
in place and were looking to enhance or refine them in some way. 

Grant objectives can also be categorized with respect to their major area of focus. 
Along these lines, the most common cluster of objectives related to grantees’ efforts to 
build electronic consumer report systems. It is widely recognized that assembling 
performance information about each program for which vendors are applying for 
eligibility and developing a user-friendly computer interface by which prospective 
trainees can access that information are mammoth and costly undertakings. In 
recognition of this, ten of the thirteen grantees specified grant objectives that related in 
some respect to building or enhancing their consumer report system. In some cases, this 
represented an effort on the part of state grantees (or state partners of local-area grantees) 
to build a major part of their statewide CRS. The grants of Indianapolis, Georgia, 
Missouri, Nebraska, North Carolina, and Pennsylvania can be classified in this way, with 
some being much more specific about the system they were building than others were. 

By contrast, several local-area grantees, including Baltimore, SELACO, and Southwest 
Connecticut, were interested in building a local CRS, either as a supplement to the state’s 
emerging system (Baltimore) or as a temporary substitute (SELACO, Southwest 
Connecticut) in recognition of the fact that the states’ systems would not be operational 
for quite some time. As a further adjunct to the CRS, two grantees (Missouri and a local- 
area partner working in conjunction with Pennsylvania) are exploring the possibility of 
establishing a customer messaging system, whereby trainees could post comments about 
the quality of the training they are undertaking. 
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The next largest cluster of objectives, articulated by eight grantees, related to 
capacity building, typically for staff. As with the group above, the grantees in this 
category varied with respect to the specifics of what they were trying to accomplish. For 
example, Baltimore was interested in hiring experts in assessment and counseling, one for 
each of its several One-Stop Centers, to work with customers and case managers to 
ensure that customers had the best information possible concerning their skills and other 
attributes so that they could make wise decisions about suitable programs of study. 
SELACO had a similar objective but a markedly different approach. This grantee was 
intending to use its grant funds to develop and refine the curriculum for a two-week 
group workshop that all prospective trainees would be expected to attend. The objectives 
of the workshop were to enable customers to support and assist each other through the 
process of making training choices, benefit from each other’s experiences and peer 
support, and develop teamwork skills through this “task team” approach. Other grantees 
focused more traditionally on developing and delivering curricula for staff training 
related to 1TA or ETP issues. Although this usually was intended to take the form of 
workshops or conferences for staff, some (Missouri and Pennsylvania) were interested in 
developing web-based staff training materials or modules. 

The third largest category included six grantees that identified an interest in 
developing or testing 1TA policies. Again, the specifics varied. One grantee 
(Indianapolis) is working in conjunction with several other local areas in the state to test 
alternative 1TA polices, as a way of gathering information about which ones might be 
most effective. One of the local areas partnering with Ohio will administer an 
“environmental scan,” again with an eye to learning about, and learning from, variation 
that is naturally occurring across local areas. 

Macomb-St. Clair has a different approach to regional variation. This grantee is 
concerned (as are many other local areas around the country) about the confusion and 
possible unwanted competition among local areas that could ensue if each local area 
within a regional labor market adopted different 1TA policies, such as with respect to 
caps on the duration or costs of training that would be funded or occupations considered 
to be in demand. If there were such variation, it is feared, customers could play off one 
local area against the others, or shop around for “the best deal.” To circumvent such 
difficulties the seven local areas that make up the Southeast Michigan Workforce Agency 
Coalition (SEMWAC) elected Macomb-St. Clair to spearhead an effort to develop 
uniform regional 1TA policies and procedures. As part of its grant activities, then, this 
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grantee hired a consultant to facilitate a workgroup of SEMWAC members that 
discussed: what funding streams should use the ITA process, what ITAs can be used to 
fund, how demand occupations should be defined, what funding or duration caps should 
be imposed, payment structures and processes, how ITAs should be coordinated with 
funds from other sources (e.g., Pell grants), and how trainees’ progress while in training 
should be tracked. It also is discussing developing a regional approval process for 
vendors submitting eligibility applications, so that a vendor would not have one of its 
programs approved by one local area but rejected by a neighboring one. 

In another example of how grant funds are being used, three grantees are 
attempting to develop software to facilitate the tracking of ITAs and ITA expenditures. 

In some cases, the software will enable customers to track their own balances (Missouri) 
and in others it is intended to be of primary benefit to the workforce area’s fiscal staff and 
case managers (Baltimore and Texas). In Texas, one local area (Golden Crescent) took 
the lead in developing ASSET, which will be used by each local area in the state to track 
ITA budget amounts and expenditures, average costs per student, training courses being 
accessed, and ITA completions. 

Three grantees (Texas, Pennsylvania, and Ohio) are using their grant funds to 
automate the vendor application process. These areas have realized that taking and 
processing paper applications is enormously time consuming and burdensome to staff. 

By automating the process, they anticipate realizing substantial resource savings. 

Finally, a number of grantees have specified objectives that are quite unique and do 
not readily fit into any of the above categories. For example, Metro Portland sees that 
synergies can be realized between ITAs and Individual Learning Accounts and is using 
some of its grant funds to recruit employers to participate in its Learning Accounts 
program. One of the local partners participating with Ohio will be testing alternative ITA 
payment mechanisms (e.g., vouchers, smart cards), and a local area participating with 
Pennsylvania will be developing a code of ethics for vendors. Other areas in 
Pennsylvania will be exploring the feasibility of integrating employers’ in-house training 
into the ETP system and developing a system to pool requests for training from different 
social service agencies so that better rates from vendors can be negotiated. 

This range of variation clearly speaks to the difficulty and complexity of 
developing I TA procedures and ETP and CRS resources, and the large number of distinct 
activities that must be accomplished and coordinated. In light of this, several grantees 
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noted how fortunate they consider themselves to be in having been selected to participate 
in this demonstration, as it has provided them with access to special funds for system 
development that otherwise would have needed to come from their regular W1A formula 
allocation. 

Summary 

The above review suggests that the grantees that were chosen to participate in the 
1TA/ETP demonstration generally had moved firmly away from contracted class-size 
training even before W1A was enacted. Most even had substantial prior experience with 
using vouchers, either as a CMA Demonstration grantee or otherwise. Due to their 
having received DOL One-Stop implementation grants during the mid- to late- 1990s, 
almost all also had made substantial progress in having built the One-Stop systems that 
W1A requires. 

These facts made their transition to the use of ITAs much easier than it otherwise 
might have been. Thus, some of our respondents noted that case managers had 
previously adopted a “coaching” approach to working with their customers and that 
customers were encouraged to exercise individual choice in choosing programs of study 
and vendors. Some sites had even begun to assemble list of vendors, with information 
about programs offered and the duration of training and its costs, as a resource for 
potential trainees to use in making their decisions. 

This prior progress notwithstanding, the transition to the use of ITAs was not 
trouble free. The most glaring problem that sites experienced was that performance 
information on vendors, consistent with the intent of the legislation, was almost wholly 
absent. Thus, substantial new work needed to be accomplished to build consumer report 
systems, as a subsequent chapter of this report will describe. Additionally, because 
training is not assumed to be the appropriate course for all adults and dislocated workers 
registered in W1A, as it might have been under JTPA, case managers in some cases need 
to change customers’ expectations. 

When we examined their grant plans, grantees identified a wide variety of ways in 
which they would be using their grant funds to help facilitate these changes. These plans 
varied in their specificity, with more narrowly focused grant plans typically articulated by 
grantees that had substantial prior experience with vouchers and who were at more 
advanced stages of W1A implementation. Most common were plans to use funds to help 
build consumer report systems. Other key objectives articulated by the grantees were to 




11-10 




noted how fortunate they consider themselves to be in having been selected to participate 
in this demonstration, as it has provided them with access to special funds for system 
development that otherwise would have needed to come from their regular WIA formula 
allocation. 

Summary 

The above review suggests that the grantees that were chosen to participate in the 
ITA/ETP demonstration generally had moved firmly away from contracted class-size 
training even before WIA was enacted. Most even had substantial prior experience with 
using vouchers, either as a CMA Demonstration grantee or otherwise. Due to their 
having received DOL One-Stop implementation grants during the mid- to late- 1990s, 
almost all also had made substantial progress in having built the One-Stop systems that 
WIA requires. 

These facts made their transition to the use of ITAs much easier than it otherwise 
might have been. Thus, some of our respondents noted that case managers had 
previously adopted a “coaching” approach to working with their customers and that 
customers were encouraged to exercise individual choice in choosing programs of study 
and vendors. Some sites had even begun to assemble list of vendors, with information 
about programs offered and the duration of training and its costs, as a resource for 
potential trainees to use in making their decisions. 

This prior progress notwithstanding, the transition to the use of ITAs was not 
trouble free. The most glaring problem that sites experienced was that performance 
information on vendors, consistent with the intent of the legislation, was almost wholly 
absent. Thus, substantial new work needed to be accomplished to build consumer report 
systems, as a subsequent chapter of this report will describe. Additionally, because 
training is not assumed to be the appropriate course for all adults and dislocated workers 
registered in WIA, as it might have been under JTPA, case managers in some cases need 
to change customers’ expectations. 

When we examined their grant plans, grantees identified a wide variety of ways in 
which they would be using their grant funds to help facilitate these changes. These plans 
varied in their specificity, with more narrowly focused grant plans typically articulated by 
grantees that had substantial prior experience with vouchers and who were at more 
advanced stages of WIA implementation. Most common were plans to use funds to help 
build consumer report s>stems. Other key objectives articulated by the grantees were to 
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use funds for staff training or other capacity building, developing or testing ITA policies, 
and building automated vendor application processes, among others. As of the writing of 
this Interim Report, it is too soon to assess the success with which grant funds were used 
and what was actually accomplished. 
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Exhibit II-l 

Grant Objectives, Planned Expenditures, and Progress to Date 
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III. CUSTOMERS’ USE OF THE ITA 



WIA has set forth a new vision for services to customers. Instead of strictly 
targeting dislocated and disadvantaged workers as JTPA did, the new law seeks to make 
employment and training services available to the entire labor force. Moreover, unlike 
JTPA, where training was the expected service for most participants, WIA permits 
training only if job seekers are unsuccessful in the core and intensive services. This 
movement through the levels of service is substantially regulated by federal, state, and 
local policies. However, once an individual is approved for training, a less regulated 
more market-based approach is promoted, in that WIA participants become customers 
who should have a great deal of latitude in selecting the training program and vendor that 
they think will best suit their needs. This choice is to be supported by staff resources and 
information on labor markets and vendor performance. 

This chapter discusses how these processes operate in the ITA/ETP sites we visited. 
It begins with a discussion of the emphasis that sites placed on training in general, and 
ITAs in particular. We use this as a departure point to explore how customers actually 
access and use ITAs at the local level. We delineate the requirements governing the path 
and actual services that job seekers use during core and intensive services on their way to 
training and an ITA, and detail the ITA decision-making process of choosing training 
occupations and vendors. The chapter then considers the services that One-Stop centers 
provide to ITA holders during and after training, and looks at how One-Stop centers 
coordinate the ITA with funding from other sources. 

In examining these issues, we note that, although WIA provides substantial 
opportunity for states to develop rules and policies affecting the use of ITAs, state 
administrative agencies generally defer to LWIAs almost entirely on customer- service 
issues. This approach reflects a deliberate state deference to local responsibility for the 
shape and conduct of participant services. 

Emphasis on Training and ITAs 

Customer access to and use of ITAs is determined by a number of important 
systematic factors that affect the emphasis that Local Boards in the demonstration sites 
place on training in general and ITAs in particular. We look at this issue through the 
prism of local policies and funding levels for training and explore how the local sites 
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balance ITAs against other forms of training. Exhibit HI- 1 summarizes some of our 
findings. 

Emphasis on Training in General 

Whether a local site has a strong interest in raising the skill levels of a small 
number of people through training or assisting a large number through job search and 
other less costly services reflects a long-running debate in the employment and training 
community. The lack of consensus on this issue was reflected as well among the sites we 
visited. Thus, some avowed the intent to continue high levels of funding for training, and 
expected to spend 50% or more of their total W1A allocation for adults and dislocated 
workers for this purpose. Others anticipate much more modest funding levels, with some 
anticipating spending less than 20% of their funds for training. 

Among the factors that entered into their decision on this issue was how to balance 
WlA’s emphasis on using training as a “last resort,” with the sites’ traditional focus on 
providing job training to meet the needs of the disadvantaged. At the one extreme, four 
of the state grantees indicated that they are actively pursuing a work- first policy, or are 
otherwise emphasizing core and intensive services. In those states, the local response is 
consistent with the state emphasis, and many fewer people are being placed in training 
than was the case under JTPA. By contrast, other states and local areas maintain a strong 
interest in training. They are able to take customers with an evident need for 
occupational skills and move them promptly through core and intensive services and into 
training. These sites are planning to train about as many participants as they did under 
JTPA. Southwest Connecticut, for example, indicated that it has a strong commitment to 
training, because training is the service that its participants have always needed. 

Baltimore and metro Portland also sought to keep their levels of training high, based on 
what they saw as the primary needs of their communities. 

The availability of funds for training was another chief consideration. Many of our 
local sites reported that they have much less money to spend on training than they once 
did, in part because of the W1A requirement to establish a One-Stop system and three 
levels of service, including core services for the entire labor force. Given this 
requirement, several states noted that there was not enough money to provide good 
quality core and intensive services and still have funds available for training. In the case 
of Texas, this funding limitation disappointed local officials who felt that the higher level 
of training under JTPA made their program successful. For example, Golden Crescent 
staff indicated that, after paying for a One-Stop infrastructure 
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Exhibit III-l 

Emphasis on 1TA and Other Training 



Local Area (Grantee) 


Emphasis on Training 


Use of Non-ITA Training 


Baltimore OED 


Planned 50% of WIA funds 
to be spent on training 


Strong interest in OJT, class- 
size, and customized training 


Indianapolis PIC 


Trying to maintain level of 
training under JTPA 


Very little OJT or other 
contract training 


Tecumseh Area Partnership 
(Indianapolis PIC) 


Planned 40% of WIA funds 
to be spent on training 


Very little OJT or other 
contract training 


Macomb/St. Clair Workforce 
Development Board 


Planned one -third of WIA 
funds to be spent on training 


Some customized training 
and OJT are planned 


Metro Portland worksystems 


Trying to maintain level of 
training under JTPA 


Emphasis on literacy and 
computer skills in intensive 
services 


Southeast Los Angeles 
County 


Training expected to be lower 
than under JTPA because of 
strong labor market 


Extensive use of OJT and 
customized training 


Southwest Connecticut 


Strong commitment to 
training, although fears 
resources will be limited 


Very little use of OJT 


Areas covered by the Metro 
Atlanta Regional Comm. 
(Georgia Dept of Labor) 


Each One-Stop center has a 
training budget 


Some class-size contracts for 
certain high volume 
occupations, retaining 
customer choice 


West Central/Sedalia (MO 
Workforce Development) 


Strong emphasis on core and 
intensive 


Some use of OJT and 
customized training 


Greater Lincoln (Nebraska 
Dept of Labor 


Planned 50% of WIA funds 
to be spent on training 


ITA is sole source of training 


Capital Area/Raleigh (North 
Carolina Workforce 
Development) 


Strong emphasis on core and 
intensive 


No plans to use OJT or 
customized training. 


Cincinnati (Ohio Bureau of 
Employment Services) 


Less WIA-funded training 
expected than under JTPA, 
but more training overall 
because of One-Stop partners 


Reduced use of OJT under 
WIA 

Emphasis on literacy and 
computer skills in intensive 
services 
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Exhibit 111-1 (concluded) 



Local Area (Grantee) 

Three Rivers Workforce 
Board (Pennsylvania Dept of 
Labor and Industry) 



Golden Crescent (Texas 
Workforce Commission) 

Heart of Texas (Texas 
Workforce Commission) 



Emphasis on Training 

Statewide work-first 
emphasis and limited 
resources; reduced emphasis 
on training 

Limited resources; reduced 
emphasis on training 

Limited resources; reduced 
emphasis on training 



Use of Non-ITA Training 

Some class-size contracts for 
certain high volume 
occupations while retaining 
customer choice; very little 
OJT 

Very little OJT ; emphasis on 
literacy and computer skills in 
intensive services 

Very little OJT; emphasis on 
literacy and computer skills in 
intensive services 



Note: The first column lists the local area we visited (with the name of the 1TA/ETP grantee in 
parentheses, if different from the local area). 



and computer systems, little was left for training. Officials in another state pointed out 
that establishing the One-Stop system imposes a substantial financial burden on the 
LWLAs. A local area’s executive director confirmed the state’s observation, noting that 
maintaining the One-Stop system drains W1A funds because other partners are not 
contributing a fair share of the infrastructure costs. By contrast, other sites are able to 
maintain ample funds for training even after they had incurred One-Stop costs, partly 
because of deliberate decisions about their local priorities and their prior progress in One- 
Stop system building (e.g., using DOL One-Stop implementation funds), and because 
other partners (such as the Employment Service) are contributing substantially to funding 
core services. Thus, Baltimore and the Greater Lincoln LWlAs are planning to spend 
50% of their funds on training, while Indianapolis has earmarked 40% for this purpose. 

Another factor that seemed to be important and led to a decreased emphasis on 
training in many sites was the strong economy. LWlAs that see employers as their 
primary customers want to get jobseekers employed as quickly as possible. These 
LWlAs frequently have jobs readily available that can be filled with no training or with 
in-house skill building on some basic business software applications through intensive 
services. Similarly, the demand for workers is so strong in its labor market that the Three 
Rivers Local Board, even when it provides training, is looking for very short-term 
interventions. Tight labor markets also reduce the demand for training from job seekers. 
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For example, SELACO, which intended to spend as much on training in its first year of 
W1A operations as it did under JTPA, finds that it is spending less than expected because 
of low demand for training by job seekers. 

ITA versus non-ITA Training 

Once local areas make a commitment to training as a service, they still have some 
options to use other forms of training besides the ITA. W1A (Section 134d4G) permits 
LWlAs to use customized training and on-the-job-training (OJT) and otherwise to 
establish contracts for training if there is an insufficient number of providers to establish 
the competitive training market that an ITA system envisions, or to fund community- 
based or other private organizations to serve participants with multiple barriers to 
employment. 1 

In general, most of the demonstration sites made little use of these contractual 
training alternatives. A number of state respondents reported that OJTs, which had been 
sharply regulated since the 1 992 JTPA Amendments, were not commonly used in many 
local areas in their states. Nor was there much use of customized or other forms of 
contract training. Only two local sites stand out as exceptions. Baltimore noted that a 
substantial portion of its funds are earmarked for non-ITA training, including employer- 
based customized training, OJT, and contract training for participants with barriers to 
employment. SELACO reported that it also planned for a substantial amount of contract 
training (although the strong economy has reduced the demand for its contract training as 
well as for ITAs). These sites felt that customized training had an important role to play 
because its unit cost was often substantially cheaper than an ITA and because the training 
could be easily tailored to meet the needs of trainees of particular types. 

Establishing Customer Access to ITAs 

Since W1A limits training to those who are unable to find a job that will provide for 
self-sufficiency, customers who receive an ITA must have gone through at least one core 
and intensive service. The local sites vary a good deal in the requirements establishing 
eligibility, movement through the three service levels, and the methods of delivering 
services. In many cases, what happens to a participant in these earlier levels of service 
has a direct bearing on the course of the individual’s ITA. 



1 However, even providers funded through contracts must meet performance requirements 
established by the Local Board. 
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Customer Flow through Core Services 

All the sites follow the statutory requirement that a customer must receive at least 
one core service before moving on to intensive services. Some of those sites, however, 
either specify a particular core service that a customer must use or establish their process 
so that a customer would almost certainly pass through that service. There was quite a bit 
of variation in what those services consist of and how comprehensive they are. For 
example, in Greater Lincoln, where ITAs are quite accessible, the core service could be 
the One-Stop orientation. In another, a customer must go through only the initial 
assessment. In Baltimore, a customer would typically begin by registering with the 
Employment Service job matching system. If a match occurs, the customer would 
receive a referral, but otherwise is scheduled to meet with a case manager who would 
help the customer to refine the job matching criteria and use self-assessment tools. At the 
other end of the spectrum, one site requires multiple core services and is very specific 
about their content. To advance to intensive services, customers generally must: 

© Attend the One-Stop orientation. 

© Take the core assessment. 

® Register with the Employment Service job matching system 

° Attend two job search workshops. 

® Apply for six jobs without an interview or attend three interviews without 
receiving a job offer. 

Needless to say, this site does not send a high percentage of people who start in core 
services on to training. 

Most of the sites we visited applied the same requirements for core services to 
adults and dislocated workers. As the basis for this, a case manager at one of the Atlanta 
Regional local sites did not believe that there were sharp distinctions between adults, who 
were mostly low- income or on welfare, and dislocated workers. She sees each group as 
overlapping on a continuum, with some low- income adults as vocationally ready as some 
lower-skilled dislocated workers. The major difference she noted was that low-income 
customers often required somewhat more staff resources because they tended to be less 
sure of what they wanted to do. 

On the other hand, a few sites had a completely different approach for each group. 
At Southwest Connecticut, for example, dislocated workers are ushered into the resource 
room and encouraged to sign up for job search workshops; at the end of the workshop 
series, they conduct a self-assessment and work on a resume and only then will meet a 



case manager in a one-on-one session. By contrast, low-income adults are encouraged to 
sign up for membership in the One-Stop center and meet with the membership 
coordinator who probes for basic information, including whether the customer has an 
interest in training. Those who have such an interest and meet other priority-group 
criteria meet with a case manager that first day, so that counseling and basic skills and 
other assessments can begin. The local area staff had previously found that, without this 
extra staff assistance, low- income adults and welfare recipients were not successfully 
navigating through the One-Stop’s self-service activities. 

Regardless of the general approach, however, all the demonstration sites, including 
those that had a more rigorous set of required core services, indicated that guidelines 
were not meant to be followed rigidly. In fact, all the sites emphasized their flexibility 
and that they treated all customers individually according to their needs. Thus, staff 
endeavor to quickly identify people who were demonstrably in need of more intensive 
services or those who may have been referred by another One-Stop partner and could 
assert or present evidence that they had used core services elsewhere. 

Because of this flexibility, there was extremely wide variation in the length of time 
that customers would spend in core services, even within a given center. Thus, many of 
our respondents were quick to note that customers who were evidently in need of more 
staff assistance could move from core to intensive services very quickly, potentially even 
within a single day. On the other hand, customers who appeared to have marketable 
skills might spend several weeks in core services, or even longer, while engaged in job 
search. 

Clearly, then, the staffs ability to identify a low- skill customer for whom job 
search is likely to be futile is critical to shortening the period in core services. The 
customers’ own motivations and preferences can be important as well. Thus, several case 
managers indicated that customers who knew they wanted and needed training could be 
moved through core services quite quickly. In one site, for example, where core and 
intensive combined ranged from three days to three weeks, the duration was largely 
determined by how quickly the customer wanted to schedule case management and 
counseling sessions. 

This substantial within- site variation notwithstanding, site-to-site differences were 
also important, including each site’s general propensity to use training. As noted earlier 
in this chapter, some sites are deeply committed to providing more training to customers, 
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and in these sites customers are moved along to intensive services more quickly than 
elsewhere, on average. By contrast, other sites are strongly committed to exhausting all 
possibilities that customers could be employable with their existing skill sets, leading to 
durations that are usually longer. For example, at one site we observed two dislocated 
workers who were still in core services after several weeks, even though they expressed a 
strong interest in computer-related training. Although these individuals were interested 
in training that would lead to careers that offered excellent employment prospects and 
high wages, this LW1A required customers to fully work through a job search before they 
would be determined eligible for an ITA. Paradoxically, in some sites with a strong 
commitment to training, discussions related to ITA decision-making, such as occupation 
or vendor choice, may occur during core services, having the effect of increasing the 
duration of core services (although they should reduce the amount of time for ITA 
decision-making). 

Another important element of each center’s own policies that to some degree was 
related to the average time that customers spent in core services was how much the center 
was inclined to emphasize self-service as opposed to staff- assisted core services. We 
observed two general approaches: sites that emphasize self-services and sites whose staff 
intervene with customers almost immediately. Among the sites that emphasize self- 
services, participants enter a resource room to register with the Employment Service and 
use on-line job search systems and computerized assessment and resume tools. 

Typically, they use these tools on their own, and the principal staff function is to help 
people in their use. In these centers, staff move about the resource room to see if anyone 
is having trouble with the equipment or software or otherwise needs help, but otherwise 
let customers alone. As a consequence, in this and similar sites, many customers may 
have relatively little contact with staff, especially if they are familiar with computerized 
information tools or are otherwise computer- savvy. 

Some sites, on the other hand, use a staff- intervention approach, with case 
managers sometimes offering assistance as soon as a job seeker comes into the One-Stop 
center. In Cincinnati, a customer meets a staff member immediately and moves into 
some form of staff- assisted core services. In Baltimore, a series of staff members 
“facilitate” a customer’s use of core services. Many sites combine both approaches, as in 
the Atlanta Regional local areas. In their One-Stop centers, self-service was the norm 
after an orientation, but anyone requesting staff assistance was immediately assigned to a 
case manager. 
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Among those sites where the customer is already working with a staff member, it is 
easy to determine whether the core job search is unsuccessful. In sites where there may 
not be a staff member who is monitoring progress, One-Stop centers may require 
customers to document an unsuccessful job search, which might include a list of job 
contacts an Unemployment Insurance claimant may have submitted to support a 
continuing claim. 

Customer Flow through Intensive Services 

In keeping with the legislation (W1A Section 1 34d3A), customers are eligible for 
intensive services only if they are unemployed and unable to obtain employment through 
core services or are employed but are in need of intensive services to attain self- 
sufficiency. All the One-Stop centers we visited follow these statutory requirements. 

With respect to those already employed, the law does not set a specific means test for self 
sufficiency, theoretically allowing LWlAs to provide intensive services to any adult. 
However, most Local Boards set their own definitions for employed adults at quite low 
levels, effectively barring services to those who are not low- income. The most common 
standard is the lower of the poverty level or 70% of the Lower Living Standard Income 
Level, which is the federal means test for low- income adult status. A few local areas peg 
their self-sufficiency level somewhat higher, such as at the Lower Living Standard 
Income Level, allowing more adults to qualify for these services. For dislocated workers 
a common standard is 100% of the worker’s wage at dislocation. Another Local Board, 
Three Rivers, sets self-sufficiency at 93% of the dislocation wage, a standard that 
excludes more dislocated workers. That board, however, was considering a substantial 
increase in the definition of adult self-sufficiency to equal a “living wage,” which might 
be in the $8-$ 10 range. 

Closely linked to self-sufficiency is the legislation’s stipulation (W1A Section 
1 34d4E) that, within the adult program, priority for intensive and training services should 
be given to those who are low- income adults or are welfare recipients when funds are 
limited. Local Boards must establish their own priority policies. Most of our sites 
formally declared a funding shortage at the beginning of the program year to ensure 
priority for the economically disadvantaged. Several also adopted additional priority 
criteria. For example, the Missouri West Central local area gives higher priority to 
people with the greatest number of barriers to employment. One site uses a geographic 
emphasis, giving priority to residents of its Enterprise Zone. Four local sites make 
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veterans a priority group. Another site establishes priority for those who lack any work 
experience, lack relevant work experience, or have the least education. 

Many local boards provide exemptions to their priority policies to permit intensive 
and training services to adults who are not low- income and may not fall into any of the 
other priority groups that were established. Thus, even where funding shortages were 
declared, case-managers are often authorized to exercise their discretion to approve 
intensive or training services for some participants outside the priority groups. Two other 
sites indicated that they did not face a funding shortage at the time of the site visit, so 
priority was not established for low- income or welfare adults. With so few customers 
unable to find employment through core services, they were able to train any individual 
who was otherwise eligible, despite their priority policy. 

With respect to the content of intensive services, all the local sites we visited 
require a full assessment and completion of an individual employment plan. Most also 
offer the normal menu of intensive services as specified in the legislation, including basic 
skills training, brush-up skills on computers and basic office software, as well as 
individualized job search assistance. But sites differ considerably in the way they 
implement their designs, often according to the emphasis they placed on training. 

In sites with less of an emphasis on training (see Exhibit IU- 1 ), job search is often 
the focal point. These sites tend to send only a small portion of their intensive-service 
participants on to training and then only after a period of concerted job searching. 

Among the common reasons given for this approach are to meet the demands of local 
employers, limited funding for training, and the statutory requirement that training occur 
only after intensive services are unsuccessful in helping the participant to find a job. In 
these sites, the assessment and individual employment plan focus on expanding the job 
search rather than increasing the customers’ skill level. In one site, for example, the first 
intensive service is a conversation between a case manager and participant to analyze 
why the job search in core services was unsuccessful. The customer then receives 
assessment and is routed into particular job search workshops to address any 
shortcomings of the earlier job search strategy. 

By contrast, in nearly half of the local sites a large percentage of participants in 
intensive services eventually go on to training. Consequently, in these sites intensive 
services revolve around planning for training, which in most cases is the ITA. The 
assessment and individual employment plan typically identify an appropriate training 
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occupation and vendor. For example, in both Greater Lincoln and Greater Omaha the 
staff had already identified those people who needed and wanted training before moving 
them into intensive services. For such individuals, the assessment and individual 
employment plan developed as part of intensive services focus on training decisions. 
Similarly, in Allegheny County the individual employment plan identifies the skills and 
courses the customer needs to achieve a career goal. 

Still other sites have expanded the menu of intensive services considerably, at least 
partly to ensure that short-term skill building can be accomplished without the need for 
training services. For example, one site emphasizes how much richer its intensive 
services became when it brought basic skills, computer literacy, and common business 
software applications into its intensive service menu. These enriched services enable 
many customers to address their skill deficits and find a job within intensive services; 
thus, relatively few are going on to training. 

Movement from Intensive Services to Training and ITA 

Most of the sites reported that they do not add any additional criteria to the basic 
statutory requirement that customers should only move to training if they are unable to 
find a self-sufficient job through intensive services and have the skills to succeed in 
training. Still, while not modifying the qualifying requirements, some sites interpret 
them very narrowly, making a very careful review for evidence that a customer has any 
transferable skills. For example, in one site staff indicated that they are cautious in 
approving training in order to avoid mistakes; thus, even though one -third of the local 
area’s budget had been set aside for training, only three customers (out of approximately 
75 enrollees) had been issued an ITA at the time of our site visit. 

Other sites seem readier to move an individual to training, while still providing a 
rigorous set of intensive services. For example, in Southwest Connecticut the decision to 
train is made while the individual employment plan is being developed, but after efforts 
have been made to identify and address obstacles to employment such as poor 
interviewing skills or lack of good labor market information. 

Choosing Training Occupations and Vendors 

Once customers have been determined eligible for training services, they confront a 
key issue in the ITA/ETP system: how One-Stop staff and procedures can help them 
make good choices. These choices need to be agreeable and beneficial to customers and 
contribute to the favorable outcomes for which Local Boards and states are held 
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accountable. In this section, we explore the heart of the 1TA system by discussing the 
resources and processes that LWlAs use to effectuate choice. 2 We look at several models 
of customer choice, the processes used to make selections and approvals, and the impact 
that vendors have on decision-making. 

Types of Customer Choice 

The essence of customer choice is present in all the sites we visited, in that 
customers are typically choosing occupations and vendors that they want. We saw no 
evidence that customers are being placed in classes contrary to their preferences because 
the LW1A had negotiated a contract and needed to fill training slots. 3 These findings 
suggest that there has been substantial progress in the sites we visited in reaching one of 
the key statutory objectives of the 1TA system. 

The real variation among sites occurs in the way choices are made rather than 
whether there is choice at all. To help explain these variations we have posited three 
models of customer choice that reflect the broad middle and the outer bounds of how 
decisions are made in the 1TA system at the sites we visited. 4 We call these variants 
informed choice, directed choice, and free choice. 

Informed Choice. In the informed choice model, which constitutes the broad 
middle ground, the One-Stop centers provide information and assistance and develop 
procedures that lead customers to make wise choices on their own. We found an 
overwhelming preference among the One-Stop centers we visited for this model. 
Throughout these sites, our respondents consistently described customers as making 
choices based on good labor market information, a realistic assessment of their abilities, 
and some knowledge of the training programs and their performance. Most of the key 
system components to support this type of decision-making are already present. All the 
states have labor market information systems available, typically allowing online access 



2 As we noted in our discussion of intensive services above, some of these decisions actually take 
place during intensive services. 

3 Two sites in large metropolitan areas reported that they retained class-size contracts for certain 
occupations for use by 1TA holders. Staff indicated that these contracts were for high volume training 
occupations such as certain computer network engineers or various medical specialists and technicians, and 
were only used for customers who clearly expressed a preference. 

4 Our colleagues at Mathematica Policy Research developed these three models for the experimental 
evaluation of 1TA outcomes described in Chapter 1. 
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in searchable formats. Assessments are required and conducted using the vocational 
counseling industry’s standard tools. Some local areas use certified counselors, either in- 
house or under contract. In addition, information about the training programs and 
vendors comes from the eligible provider lists that are mostly now available online. Case 
managers also frequently supplement the information available from the ETP list with 
their personal knowledge of vendor capabilities . 5 In general, we found that, when 
customers had good ideas based on sound labor market information, staff were likely to 
provide a wide scope for free choice. Overwhelmingly, then, our interviews and 
observations of counseling sessions suggested that the customer is the real decision- 
maker. As one case manger in Southwest Connecticut noted, “We have to respect the 
customers’ knowledge of themselves (because, ultimately) they are the ones who have to 
show up (for the training).” 

Nonetheless, front-line staff play a key role in guiding or facilitating choices. Case 
managers consistently describe themselves as guides, facilitators, or information brokers, 
and one site even includes the term “facilitator” in all the job titles for its front-line staff. 
Case managers thus clearly play a pivotal role in ensuring that the customer’s choice is 
actually informed by good information. The role generally appears to be a real 
compromise between expertise and facilitation. One case manager in the Atlanta area 
describes this balancing role in working with a customer who may be making a poor 
decision as follows (below is a paraphrase): 

We work extensively with customers to persuade them about which vendor 
would be suitable, in terms of the curriculum, pedagogical approach, and the 
types of ancillary services available at the school. However, if a customer 
insists on choosing a particular vendor — even if we think there is a more 
suitable vendor — we normally approve the request, but remind the customer 
that there will be no second chance (at further training). 

Consistent with the facilitative role, several sites substantially increased the level of 
responsibility for front-line staff when compared with their roles before a voucher system 
was introduced. Thus, for example, administrators in Baltimore noted that their front-line 
staff were no longer the “paper pushers” they had been under JTPA, but are now 
empowered to work with customers to respond to their unique needs. 



5 At the time of the site visits, only two of the states had consumer report systems operational, with 
vendor performance information available. Thus, case managers’ knowledge of different vendors and their 
performance might have been used more heavily than might otherwise have been the case. 
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Directed Choice. Directed choice provides the same types of labor market and 
vendor information to the customer, but in this model staff must work especially hard to 
ensure that the customer makes a decision that is heavily influenced by the staffs 
professional judgment. Only two sites appear to be practicing directed choice for all their 
customers. In these sites, case managers try to control the terms of the decision-making 
to a greater degree than in the informed choice sites. Staff work to convince customers 
that their professional judgment is preferable to the vagaries of the customer’s less 
informed judgment. They may be very blunt in suggesting to their customers what 
program to select and where they should go to school. By way of justification, an 
administrator in one site noted that “people do not always know what is best for them.” 

Under some circumstances a directed choice model is also used in centers that 
generally practice informed choice. For example, one case manager who worked mostly 
with current or former welfare recipients indicated that many of her customers did not 
readily understand how to use the decision-making tools or how to interpret their output, 
so she has to take a much more directive role than she otherwise would. Elsewhere, case 
managers often play a key role in dissuading customers from their choices when they 
come into the center with ill-informed training ideas — occupations with poor prospects 
for self-sufficiency or vendors ill-suited to their learning styles. The high level of staff 
intervention with such customers effectively reduces the scope of independent decision- 
making that is available to most others. 

Free Choice. In a free choice model, customers can access labor market data, 
vendor information, and assessment tools at their discretion, but are generally left free to 
make their own training choices (so long as the program of study is for a demand 
occupation and the vendor is on the eligible provider list). While no sites use a standard 
free-choice model exclusively, it does occur quite frequently for customers who know 
exactly what they want before they come into the One-Stop center. These customers 
often have very specific training plans with an occupation and a vendor already picked 
out at the outset. Where the soundness of plans can be confirmed based on good labor 
market information, the vendor’s suitability, and a realistic assessment of the individual’s 
own capabilities, these customers typically get their choice with a minimum of staff 
intervention. 

Process Used in Making Choices 

In addition to requiring assessment, providing access to labor market information, 
and offering professional guidance and facilitation by staff, many of the sites created 
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processes that also help customers make a better- informed choice. Indeed, some of these 
processes, which include workshops and participant research, represent the means by 
which informed choice is ensured. In addition, a formal approval process is sometimes 
used, as a way of ensuring that decisions are justifiable and to benefit the administration 
of the LWLA by assuring that ITA funds are being spent wisely. 

Workshops . Several sites use workshops as an important tool in assisting 
customers in making choices. For example, Baltimore uses a training workshop for all 
customers going on to training. Within that workshop customers leam about their 
training alternatives such as ITA, OJT, or customized training. If they choose the ITA 
method, they attend an ITA workshop, which helps them to prepare their application for 
an ITA that identifies and justifies their selection. Similarly, Southwest Connecticut also 
uses a workshop to prepare customers to conduct active research and teach them how to 
interpret performance information. In a very different approach, SELACO uses a two- 
week, peer- managed workshop where customers work together in a team to help each 
other to select appropriate programs of study and vendors. 

Participant Research. A key part of informed choice is to have customers 
research their intended field of study. Most of the demonstration sites implement this by 
specifically requiring or encouraging customers to conduct field research on both training 
occupations and vendors. This typically involves having customers visit vendors to talk 
with staff and observe the facilities. Several also add a requirement that customers must 
either talk with current or former students of the program, interview employers who 
would potentially hire trainees in that occupation, or both. For example, the Pittsburgh 
One-Stop center requires its ITA candidates to interview at least two schools, two 
students who had completed the program, and two potential employers. The case 
manager at another site noted that her LWlA’s research requirement was very effective in 
“bringing back to earth” participants who had come in with poorly thought out ideas. 

Some case managers also felt participant research increased the participant’s commitment 
to the eventual choice, and enhanced the case manager’s knowledge base. 

Approval Process. Several sites make a formal approval process a part of an effort 
to promote informed choice, while also serving as a means to assure LWLA management 
that customers are making defensible training decisions. In sites that have such a process, 
potential trainees must submit a formal application or make a presentation before a 
review panel with the authority to approve the ITA. For example, Baltimore’s approval 
process is as follows: 
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In Baltimore, prospective trainees must prepare a written justification for their ITA, 
including reasons for wanting the training, evidence that the training is for a 
demand occupation and that the individual has the skills needed to successfully 
complete the training, and a record of research that the participant is required to 
undertake, including from interviews with vendors, former trainees, and employers. 
Prospective trainees then present this justification to a panel of the city’s One-Stop 
center directors who are authorized to approve the ITA. Those whose application is 
deemed incomplete or unsatisfactory must conduct additional research and prepare 
a new application, with the assistance of the case manager, until the review panel is 
convinced that the choice is a wise one. 

Similarly, Golden Crescent has its participants present an oral proposal to an ITA 
committee justifying the career and vendor choices and identifying expected outcomes in 
placement and wages, barriers to completing the training, and a financial plan that 
demonstrates that training and living expenses can be met. 

Local areas in one grantee state, Pennsylvania, are also creating additional 
decision-making tools to assist customer in making better quality decisions. Philadelphia 
is developing a written customer manual for its ITA process that lays out the participant 
rights and responsibilities that underlay an informed choice. The manual will be 
available to other Pennsylvania local areas to customize according to their own 
procedures. Another local area is supporting an Internet start-up company that will 
provide an Internet messaging site on which trainees could post comments on their 
training providers, including prerequisites, curriculum, and instructor quality. 

Staff Training and Organization to Support Decision-Making 

Because case managers ostensibly assume new responsibilities under the various 
ITA models, one might expect a substantial need for staff training and capacity building. 

In fact, several of the grantees proposed grant activities related to staff training. Overall, 
however, we found that relatively little formal training on ITA systems or the case 
manager’s ITA role had yet occurred. Respondents suggested two reasons for this. First, 
because many of the sites were already using some form of vouchering under JTPA, the 
beginning of the ITA system did not change the case manager’s role much, if at all. No 
site indicated that it had to retrain its front-line staff systematically in order to provide its 
particular type of informed choice. Second, many sites are still gearing up for the ITA 
and have plans for training in the future once their automated systems are in place. For 
example, Georgia and North Carolina expect to provide training on their integrated career 
decision and eligible provider list/consumer report system once the systems are fully up 
and running with performance data. 
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The training that has occurred has been a mix of state- sponsored and local in-house 
training. For example, a number of sites, such as Indianapolis and Metro Portland, 
provided in-house training on their own ITA policies and procedures. Several other sites 
reported that there was some coverage of ITA rules and procedures in their general WLA 
training, either provided in-house, directly by state staff, or by the state training institute. 
Several sites provide training to help case managers play a generic facilitative role, 
because the sites are trying to embed facilitation in the entire range of customer 
relationships. The Baltimore One-Stop center, for example, wants staff to help customers 
to make decisions rather than to tell them about rules and procedures. The center uses 
role-playing to equip staff to handle specialized situations creatively. As part of its grant, 
it was also hiring assessment specialists who would not only work with customers, but 
provide guidance and training to other staff at the center. Similarly, the Three Rivers 
Local Board paid for facilitator training at a local university for any interested case 
managers. 

Apart from capacity building, another key staffing issue as sites move to implement 
ITAs is how they organize staff to assist customers. We found two general approaches. 
Some centers use a single point of contact, while others rely on specialization. In the 
former approach, a single case manager stays with the customer throughout training, 
intensive, and staff-assisted core services. The advantage of this approach is that the case 
manager builds up knowledge of the customer and a personal relationship. Capital Area 
(Raleigh) and Golden Crescent are examples of One-Stop centers where the case 
managers build a personal rapport to aid in the decision-making, even though these two 
sites differ considerably in the way in which case managers provide information and the 
degree of influence over customer choice. Some sites that use a single case manager 
have specialized assessment units or contracts with outside firms for assessments. In the 
second approach, One-Stop centers use specialists on staff in working with customers. 
This approach enables case managers to develop greater expertise in the subjects to 
which they are dedicated. Baltimore has the clearest division of labor, with case 
managers who specialize in job search during core and intensive services, and others who 
focus on assessment and career planning. These staffing patterns and relationships cut 
across both the informed and directed choice models. 

Limits on Customer Choice through Local Policies 

Regardless of the customer- choice models that local sites selected, nearly all 
exercised their authority to set limits, predominantly to ensure that funds would be 



available to support as many trainees as possible. The most common are limits on the 
1TA dollar amount, duration, and permitted uses of the funds. All sites also implicitly 
narrow choice by identifying occupations in demand in the local economy,, for which 
training will be supported. Other types of limitations such as residency preferences or 
formal linkages to local economic development are uncommon. Some of these limits are 
summarized in Exhibit HI- 2. 

Dollar and Time Limits. Nearly all of the sites set a dollar limit, or cap, on the 
total amount of money that a participant could receive for an ITA. Of the 18 local areas 
we visited, 1 6 use caps. The average cap among those 1 6 areas is about $5,000, ranging 
from a low of $1,700 to a high of $10,000. Three sites provide authority for staff to 
authorize ITAs that are higher than the local cap, if special circumstances warrant. In 
setting their caps, two sites use their community college as a reference. For example, in 
Macomb- St. Clair the Local Board pegged the cap at the cost that the local community 
college charges. In North Carolina, the state requires that any cap be set above the cost 
of training at the community college in order to encourage competition in that state for a 
training market that is already dominated by community colleges. 

Nearly all the local sites, including those with no cap, set a two-year duration on the ITA. 
Among the exceptions, one site limits its ITAs to one year, while another sets a monetary 
cap for one year but allows trainees to request a second year of training with the same 
dollar amount. Three sites have no limitation on duration, although one of those 
indicated that, in practice, most of their ITAs are for one year and there is a virtual two- 
year limit. 
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Exhibit HI-2 

Local Limits on Amount or Duration of ITA 



Local Area (Grantee) 


Dollar Cap 


Duration Limit 


Baltimore OED 


$7,000 


1 year 


Indianapolis PIC 


$2,500 


2 years 


Tecumseh Area Partnership 
(Indianapolis PIC) 


None 


2 years 


Macomb/St. Clair Workforce 
Development Board 


$5,000 


None 


Metro Portland worksystems 


None 


None 


Southeast Los Angeles 


$5,500 (staff can justify 


None (however, ITAs are 


County 


amounts above the cap) 


rarely for more than 1 year) 


Southwest Connecticut 


$3,000 


1 year 


Areas covered by the Metro 
Atlanta Regional Comm. 
(Georgia Dept of Labor) 


$8,000 


2 years 


West Central/Sedalia (MO 
Workforce Development) 


$1,700 


2 years 


Greater Lincoln (Nebraska 
Dept of Labor 


$5,000 


2 years 


Greater Lincoln (Nebraska 
Dept of Labor 


$4,000 


2 years 


Capital Area/Raleigh (North 
Carolina Workforce Develop- 
ment) 


$5,600 (State policy that ITA 
must exceed cost of training 
at community college) 


2 years 


Cincinnati (Ohio Bureau of 
Employment Services) 


$5,000 


None 


Three Rivers Workforce 
Board (Pennsylvania Dept of 
Labor and Industry) 


$ 1 0,000 (considering a policy 
to vary cap to promote high 
wage/high growth occ.) 


2 years 


Golden Crescent (Texas 
Workforce Commission) 


None 


2 years 


Heart of Texas (Texas 
Workforce Commission) 


$5,000 


2 years 


Note: The first column lis ts the local area we visited (with the name of the ITA/ETP grantee in 
parentheses, if different from the local area). 
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Permitted Uses for the ITA. Nearly all the sites permitted ITAs to be used for 
tuition, fees, books, uniforms, and equipment. Only one site limited the ITA to the 
tuition, while one other prohibited its use for fees. Only one site included supportive 
services within the ITA. One of the sites that kept its supportive services separate still 
linked the amount of those services to the size of the ITA. In that site, supportive 
services could not exceed 20 percent of the ITA amount. Yet another site tracked 
supportive services along with the ITA, although the funds came from separate accounts. 

Demand Occupations . While the law requires that all training occur in demand 
occupations, sites defined this requirement in different ways. Some relied on a list of 
demand occupations developed by the state; others used locally developed lists. A few 
had no formal lists at all, but instead relied on the case managers’ own knowledge of the 
local economy. Sites with formal state or local lists also varied in whether exceptions 
could be made to fund training not on the list. Most would allow such exceptions if the 
customer could demonstrate that a job would be available once training was complete; a 
few allowed no waivers whatsoever. 

Several sites added additional restrictions to enhance the economic development 
potential of their ITA system. For example, one site is planning to link ITA amounts to 
important growth sectors in the local economy, as follows: 

The Three Rivers Local Board may consider a new policy in which the 
amount of the ITA would vary with the occupation chosen. Under the policy, 
the Local Board would reserve ITAs with the highest permissible funding 
amounts for training in jobs that are considered critical for regional economic 
growth. The absolute maximum would apply for the occupations that the 
Local Board believes are most important for the local economy and that pay 
the highest wage. Other occupations would be capped at a stipulated 
percentage of the maximum amount, varying with the relative importance of 
the occupation to economic growth in Pittsburgh and surrounding Allegheny 
County and its ability to lead a customer towards a living wage, which is 
expected to be in the $8-$10 range. 

Residency Preferences. There are also a few residency preferences to access 
ITAs. Even though W1A removed JTPA’s residency requirement for service to 
economically disadvantaged adults, several sites within the demonstration have decided 
to impose residency preferences on the issuance of ITAs. For example, the Atlanta 
Regional local areas give preferences to residents of their respective LWlAs or to 
dislocated workers formerly employed in those LWlAs, on the grounds that serving those 
outside the area comes with no additional funding. 



Services P rovided to Trainees during and after Training 

Services during Training 

The extensive investment in the ITA by both the participant (time and effort and 
opportunity costs) and the LWIA (staff time and training outlays) raises the importance 
of ensuring that the customers are successful in their training. One element of this is 
maintaining good connections between the One-Stop center and customers who are in 
training. For the customer, a variety of counseling services may be beneficial or 
necessary to assure training completion. The One-Stop center staff, in turn, will want 
assurance that the participant is attending classes and otherwise making progress in the 
training. Maintaining these connections typically involves the participant, the One-Stop 
center case manager, and the vendor and its staff. All the sites earned out these functions 
in some way, but they differed considerably in the types of services offered or required 
and in who carried out these various responsibilities and services. 

All but one of the sites want trainees to keep in periodic contact with their case 
managers during their training period to verify attendance or to meet with the case 
manager. The most common period used is a monthly meeting or report. However, sites 
differ in whether the responsibility for contact lies with the trainee or the case manager. 

In some sites, the trainees need to check in to file an attendance or a broader progress 
report and could then elect to see a case manager to discuss any concerns that they have 
with their training. Trainees are also supposed to report any problems that might 
interfere with the completion of their training program, such as health or family 
emergencies. The Greater Omaha site wants contact on a more frequent basis. Trainees 
there have to bring in attendance reports signed by the training instructor at least every 
two weeks. This visit coincides with the time customers pick up bus passes, an important 
supportive service for many of them. When the customers come in, the case manager 
gets a chance to talk with them informally. 

Case managers have parallel responsibilities to monitor a trainee’s progress. All 
sites have their case managers monitor progress in the training in some form. In those 
sites that require a personal visit, case managers meet with their customers to solve 
personal problems that might interfere with completion of the program or to review 
supportive services needs. In some sites, such as Southwest Connecticut, where the 
trainee is not required to come in, the case manager checks monthly with a trainee to 
learn about attendance and any concerns with the training. The case managers may even 
make a home visit, if there was a concern about attendance. Another site sends trainees 
periodic surveys mailed to their home; this approach was developed partly because case 
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managers have very heavy caseloads, which make routine personal contacts infeasible. 
Demonstrating another approach, the Georgia 1TA coordinator is planning to create an 
Internet- based network to provide study skills to trainees. This system, which will be 
part of the integrated career exploration and consumer report systems, will be available to 
all trainees in the state. 

Vendors also provide important services during training. Nearly all have financial 
aid case managers to assist with applications for Pell grants and other forms of financial 
support and a counseling staff to provide training in study skills or help their students 
with general problems. Most of our sites required their vendors to report progress or to 
identify when students have particular problems. Five sites specifically require the 
vendors to provide periodic progress reports, most often monthly. 

Many of the ffont-line staff we spoke with noted that active counseling is much 
more prevalent among the proprietary schools than public institutions, which are 
predominantly community colleges. The community colleges usually have similar 
counseling available, but usually the student must take the initiative to use such services. 

The proprietary schools, on the other hand, are very active in bringing those services to 
their students. The proprietary schools are also generally better at monitoring student 
progress and reporting to the One-Stop centers than the colleges. Baltimore staff, for 
example, noted that their proprietary schools provide greater levels of support because 
they are much more concerned with demonstrating the institution’s high performance. A 
case manager in Jonesboro (who herself is an employee of a college that operates the 
One-Stop center and is a training vendor) spoke about how much better the proprietary 
schools are at reporting to the case manager information about students who are 
experiencing difficulties. 

However, despite these general patterns, in some places the One-Stop staff reported 
that the community college also does a good job of communicating progress and 
supporting student progress. For example, in Greater Omaha and Greater Lincoln, both 
of which have few private training schools, the community colleges provide good 
counseling services and a host of other supportive services, such as on-site child care and 
support groups for people entering non-traditional employment. 

Placement Services 

WlA's performance management requirements make good placement services 
extremely important for all One-Stop customers, including ITA holders. The sites are 
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fairly evenly divided in making the One-Stop center, the vendor, and the customer 
responsible for placement. Some of the One-Stop centers we visited want the customer 
whose training is ending to come in for job search workshops, counseling sessions, 
meetings with job developers, and help in preparing a resume, and to use other core and 
intensive services. For example, Metro Portland has the case manager refer the job 
seeker to an in-house job developer. Capital Area (Raleigh), which makes its case 
managers responsible for outcomes, asks its trainees to come to the One-Stop center two 
weeks before completion of their training to start using job search resources. One-Stop 
staff there note that, in the current tight labor market, it is very easy to place workers who 
have the up-to-date skills provided by their training. 

More of the sites, however, rely on placement by vendors. In Greater Lincoln, 
Greater Omaha, Pittsburgh, Allegheny County, and the Atlanta Regional local areas, the 
Local Boards require the vendors to assist program graduates with getting jobs. For 
example, a technical training school in the Pittsburgh area has full-time job developers 
who get bonuses for exceeding placement targets. The vendors in the Atlanta Regional 
local areas predominantly have formal placement programs, some of which we describe 
below: 

The vendors in the Atlanta area use a variety of means to provide placement 
assistance. One vendor forms job clubs and has job developers on staff. One 
computer training school that serves predominantly low- income, minority 
students integrates job search into its curriculum and makes all staff 
responsible for placement outcomes. On the other hand, another computer 
training school that serves predominantly upper income dislocated workers 
among its W1A customers facilitates self-directed job search using an 
informal network of the school’s graduates. As an adjunct to this vendor 
activity, one Atlanta Regional local area case manager bring his customers to 
the One-Stop center 60 days before completion to begin job search. 

As with counseling services, proprietary schools generally seem more proactive in 
helping trainees with job placement than public colleges are. 

Several sites give the customer the choice of whether to rely on vendor or One-Stop 
placement services, taking advantage of the customer’s obvious interest in prompt and 
remunerative employment. SELACO notes that most of its trainees take advantage of 
vendor services. Nevertheless, case managers encourage their customers to come in if 
they think that they need extra help. Regardless of who is primarily responsible, all the 
One-Stop centers indicate that their full job search facilities, from core and intensive 
services, are available to those who complete training. 



O 
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Coordinating the ITA with Other Services 

Seamless service to customers is an expected feature of the integrated system 
intended by W1A, as is the leveraging of funds from multiple sources. Thus, in order to 
understand the full array of resources and services that trainees are accessing, we must 
understand how W1A funds are used in conjunction with funds from other sources. In 
fact, in most of the demonstration sites, the ITA itself is only one part of a complex 
mosaic of funding sources that contributes to training of ITA holders. These sources 
include Pell grants from the U.S. Department of Education, trade- related dislocation 
programs through the unemployment insurance system, vocational rehabilitation state 
training or higher education grants, and welfare sources, to name the most common. 

With respect to service integration, we found that co-enrollment and other forms of 
collaboration were highly developed in some sites but weak in others. At the one 
extreme, the two sites in Indiana were notable for an especially high degree of 
collaboration across the various One-Stop partners. Thus, Indianapolis had about one- 
third of its ITA participants co-enrolled in welfare-to-work, vocational rehabilitation, or 
Trade Adjustment Assistance. Similarly, Tecumseh Area Partnership, also in Indiana, 
had quite a few ITA holders enrolled in vocational rehabilitation. At the other extreme, 
many sites had little or no co-enrollment or active collaboration. The predominant 
explanation for lack of collaboration was that their One-Stop systems were still being 
built, and they expected greater collaboration in the future. 

Of the One-Stop programs, the trade-related dislocations programs, including the 
Trade Adjustment Assistance and the North American Free Trade Agreement- 
Transitional Adjustment Assistance programs, are perhaps the most successful in 
obtaining coordinated enrollments. Both of these programs can pay for training expenses 
and income support while a customer is in training. The latter feature makes these 
programs highly desirable for dislocated workers who are interested in training. Very 
often, because the ITA could be committed before the trade amount was approved, the 
ITA was usually the initial payer and the LW1A received reimbursement once the trade 
amount was approved. Case managers at Allegheny County pointed out that the 
incentive to train was so strong and knowledge of the program was so widespread in the 
community that nearly all eligible dislocated workers come in specifically for training 
services. In this county, co-enrollment is mandatory, and the ITA amounts are 
coordinated with services provided through trade programs and other available funding 
sources. Thus, DOL’s efforts to coordinate the major trade programs with the general 
training programs, which it has been promoting over the last several years, appear to be 
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having some success. There were exceptions, however, including in the several sites, 
such as the Atlanta area, that indicated that co-enrollment was not occurring because they 
were experiencing little trade impact. 

Coordination with the Pell grant is more common still, which occurred at virtually 
all sites. 6 In managing this coordination, these sites generally adhered to WlA’s 
requirement that Pell funds, among others, should be spent first. However, because the 
ITA amount could be determined and committed immediately, in general ITA funds are 
typically used to begin a training program and then Pell grants are used to provide 
reimbursement from the training vendor once the Pell grant was awarded. Sites either 
considered an estimated Pell award in preparing the ITA, as in Indianapolis and Capital 
Area (Raleigh), or the eventual grant effectively reduced the amount of the ITA, as in 
Cincinnati, Baltimore, and West Central (Sedalia). Several other sites paid for tuition 
through the ITA but used the subsequent Pell grant to defray the cost of supportive 
services. 

Personal financial contributions were also a common form of additional support. 
Many of the sites indicated that they expected a personal financial contribution from the 
trainee wherever the ITA amount or other financial resources did not cover the cost of 
training. These amounts are figured into a financial plan that the sites required 
participants to prepare as part of the individual employment plan. Southwest Connecticut 
is perhaps the most vigorous site in obtaining contributions as a matter of policy. The 
Local Board believes that a personal financial contribution will increase a participant’s 
persona] stake in the training and the outcomes. Although not widely used, that LW1A 
also offered low-cost loans sponsored by private companies and corporate- funded 
scholarships to ensure that there would be additional resources. 

State programs were another important source of funds. For example, in the 
Atlanta Regional local areas, the presence of a state scholarship program funded through 
lottery revenue is central to their ITA strategies. These scholarships have no income 
restrictions and are thus available to virtually all residents. They must be considered the 
first source of payment for training, so residents must apply for these scholarships as well 



6 The Pell legislation contains a provision requiring that other federal financial aid programs not 
consider an individual’s eligibility for Pell when computing available resources for that person. At the time 
of the site visits, DOL and the Education Department have issued regulatory guidance that coordination 
between the two funding sources should occur. 
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as for Pell grants. The combination of these funding sources makes the ITA a relatively 
small player in the training marketplace. Pennsylvania also has a state higher education 
assistance program that could offset the ITA. 

Summary 

One of the key reform principles that DOL has developed for the implementation of 
WIA is increased customer choice. This is clearly evidenced in the ITA/ETP system, 
which provides for qualifying good training vendors and explicitly guarantees the right of 
customers to choose from among those who qualify. Yet, the system also sets that choice 
within a regulatory framework to assure that Congressional policy is carried out and 
ensure that, when funds are limited, resources are available to serve those whose needs — 
low income workers and welfare recipients — have long been recognized. This regulatory 
framework effectively constrains choice to some degree. 

We observed that the policies, practices, and procedures that local areas used in 
implementing the system were quite effective in creating a balance between these two 
tendencies. We observed that each tendency is moderated by a major factor that brings it 
closer to its counterpart. First, customer choice in the ITA system is not the unrestricted 
choice of the commercial marketplace. Rather choice under the ITA system is informed 
by high quality information and reasonable guidance by professional staff. We observed 
in all sites that customers were making choices based on the use of labor market 
information and career guidance systems, staff guidance, and other resources, including 
their own field research. Second, the regulatory framework, which some respondents 
noted is as constraining as that under JTPA, is tempered in all sites by considerable 
flexibility. Our findings on the path that customers follow through pre-ITA services, the 
selection of training occupations and vendors, and the services during and after training 
clearly show a high degree of flexibility across all sites. 

Another major factor that enabled the demonstration sites to create a more 
harmonious ITA system was their experience at it. Many of the sites indicated that they 
did not have to change many of their practices, policies, and procedures from those they 
used under JTPA. A few sites said that their ITA systems were virtually no different 
from how they approached training under JTPA. Others indicated that establishing an 
ITA was an additive process: they were able to add certain elements to existing practices 
o- • 'icies that were consistent with the ITA system. 
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IV. DEVELOPING THE ELIGIBLE TRAINING PROVIDER LIST 
AND CONSUMER REPORT SYSTEM 



Two key elements of the training system envisioned by WIA are, first, that 
customers should be empowered to make decisions regarding the training program and 
vendor that suit them best, and, second, that, for this to be meaningful choice, it should be 
supported by ample and relevant information made available by the One-Stop system. As 
we have discussed in the previous chapter, centers were actively working to provide 
information to customers to promote their informed choice, by making labor market data 
available, providing assessment and counseling, conducting workshops, and having 1TA 
holders conduct field research with former trainees, employers, and vendors. Also 
critical, according to the WIA legislation (e.g., Section 134d4F), is that customer choice 
will be supported through an Eligible Training Provider (ETP) list and Consumer Report 
System (CRS) that will: a) identify those programs and vendors that have been found by 
the state and local workforce board to meet acceptable standards of quality, b) provide 
information in a readily accessible format about each program’s costs, c) provide 
information on vendors’ performance that will permit comparisons across programs with 
respect to program completion and job placement rates of recent trainees (both WIA- 
funded trainees and others) and their wages, among other things, and d) provide other 
information that may be relevant in helping customers make wise choices. 

According to the vision embedded in the legislation, the ETP and CRS are the 
linchpin of the 1TA system, serving as key resources to promote the full information that 
is necessary to support customer choice. Providing these tools helps assure local areas 
and states that they provide choice to customers while still meeting their accountability 
requirements. However, their development has proven to be costly and time-consuming 
undertakings, as we shall see below. In fact, most of our case-study sites are still 
developing the details of these systems. For instance, with respect to the ETP, most have 
yet to make decisions regarding performance benchmarks for subsequent eligibility or 
work out the details of how vendors’ performance will be measured. The development of 
the CRS has proven to be even more difficult; only a few sites had an operational CRS, 
populated with reasonably complete data, at the time of our site visit, and many sites 
expect that it will be quite a while before they do so. Thus, the Final Report of the 
evaluation, due in early 2002, will be able to provide much more information than we 
have available now about what these systems look like, how they were constructed, and 
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how they work in practice and are used by customers. In this chapter, we provide an 
introduction to these issues by discussing some of the considerations that sites are using 
in building their ETP list and CRS, how they established initial eligibility, and their 
preliminary plans for subsequent eligibility and performance measurement. 

Design of Information Systems 

At the time of our site visits, all states had developed an ETP list that had been 
distributed to One-Stop centers for their use. These lists took different forms, reflecting 
states’ unique visions for their systems, and they were disseminated in different ways. 

Vision of the Eligible Training Provider List 

A primary question that states had grappled with was determining what purposes 
and audiences the ETP list should serve. According to the WLA legislation, the ETP list 
is to serve primarily two functions. First, as the name implies, it constitutes a list of those 
providers whose programs may be considered by adults and dislocated workers who are 
undertaking training funded by WIA (WIA Section 134d4C). Second, the list is to serve 
as a resource for the universal customer who, through core services, is interested in 
conducting research on training providers in the state (WIA Section 1 34d2F). 

There is potentially some tension between these two objectives. To meet the first 
objective as efficiently as possible the list could be restricted to only those providers 
eligible to redeem an 1TA. In this way, ITA holders can examine the universe of 
providers from whom they can undertake training with the least confusion. By contrast, 
the universal customer might be interested in researching a more comprehensive list of 
vendors, including those that may not be ITA eligible. How programs managed these 
varying goals differed among the sites we visited. 



Just less than half of the demonstration states gave primacy to the first of these 
objectives, by developing an ETP list intended primarily to meet the needs of WIA 
training customers. In these states, the ETP list consists exclusively of WIA-eligible 
training programs and providers. The logic that these states followed was that similarly 
structured lists containing information on other programs and providers already existed to 
meet the needs of customers in core services. For example, for the sake of expediency 
Pennsylvania was focusing on creating a narrow ETP list that would include information 
only for ITA-approved programs, on the grounds that doing otherwise would be 
duplicative with a more inclusive list already available from the state Department of 
Education. Sometime in the future, this state reasoned, these two lists could be merged to 
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create a comprehensive list that identified which vendors were WIA approved, but in the 
short-term it felt that its efforts would best be focused on the more limited objectives at 
hand. Another reason for developing a restricted list was to provide a strong incentive for 
vendors to apply for eligibility. According to this logic, vendors would want to apply for 
eligibility for their programs to take advantage of the marketing opportunities that the 
ETP list offered. 

By contrast, the remaining states we visited developed a broader ETP list that was 
intended to be a multi-purpose resource. In these states, the ETP list was opened up to 
include non- WIA eligible providers for several reasons. First, they wanted to ease the 
workload for their cross-trained staff, by creating more efficient search tools that are easy 
to use. Rather than different agencies having a unique list for its own purposes or having 
case managers search multiple lists with different levels of certification or types of 
approval, a single list (with indicators denoting which vendors had been approved for 
different purposes) would enable staff members to access all the training options for their 
clients quickly and with ease. For example, in Ohio Works centers, case managers are 
cross-trained and serve multiple programs for which they are responsible for training 
referrals. The database they have developed provides a comprehensive listing of training 
programs in the state while indicating whether or not the program is ITA approved and, 
in some cases, whether it is certified or approved for other funding sources. With a 
comprehensive training provider list, case managers are thus able to access one site for 
information about training vendors, and customers are able to access this information to 
make informed training decisions in a timely fashion. 

A second consideration for those developing an inclusive list was to maximize the 
use of the WIA administrative funds that were expended in building the list to begin with. 
Faced with limited funding for the development of their One-Stop infrastructure, and the 
high costs of developing the computer interface for the ETP and CRS, states want to 
leverage their investment in compiling data and developing software by having the 
resultant tools serve multiple purposes. This consideration becomes especially important 
in states that are anticipating a relatively low flow of WIA participants into training, 

4 

because in this circumstance a relatively small number of persons would benefit if the 
ETP list were narrowly restricted. By contrast, including a larger number of providers 
and programs in the CRS justifies the expense of creating this new tool. 

Related to this, states developing an inclusive list wanted a tool that would bridge 
the gap between ITA-funded training and training that might be funded from other 
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sources. Emphasizing the principles of universal access and customer choice, these states 
chose to make the ETP list a more comprehensive training source for all customers in 
including non-WlA eligible training programs. This inclusive list serves as an important 
resource for all individuals looking for training. Moreover, such a list could serve as a 
resource for employers as well. In Georgia, for example, state representatives sent out 
notices to small and mid-sized employers encouraging them to use the list for their 
incumbent-worker training. These customers would not necessarily have had a use for 
the CRS if non-WlA eligible training programs were not included on the list. 

Finally, another advantage of a broader list is to provide an incentive for vendors to 
seek 1TA approval. This explanation was paradoxical given that states adopting a 
restrictive ETP list also provided this as a rationale for their strategy. Both groups 
recognized how difficult it might be to encourage vendors to go through the trouble of 
filling out the required paperwork and collecting the necessary performance data to seek 
eligibility for a program, given that they could generally expect only a small customer 
flow of 1TA holders. Thus, states wanted to emphasize the advertising advantages that 
the vendors’ inclusion on the list would offer. Some states felt that restricting the list 
only to 1TA eligible providers would best serve this purpose. By contrast, others took the 
opposite approach. For example, the vision expressed to us by respondents in Michigan 
was that any vendor could be included on the list, whether or not it was ITA approved, 
but each must submit essentially the same paperwork and information about its programs, 
including some performance data. Given this, the marginal burden of applying for ITA 
eligibility was expected to be very modest, which, it was hoped, would induce vendors to 
apply. 



These strategies articulated by both groups of states represented their vision of how 
the ETP list could best strike a balance in meeting the needs of ITA holders, other 
potential trainees, case managers, and other stakeholders. However, pragmatic 
considerations were important as well. Thus, in some cases, the strategy followed by 
states for the ETP list was affected by their overall progress towards full implementation 
of W1A. The July 2000 W1A compliance date factored greatly in the planning of all 
states. For example, one state modified an existing list of TANF vendors to serve as the 
ETP list. Not only did modifying an existing product seem like the best way to make use 
of existing resources, it was also very expedient because state representatives were 
operating so close to the W1A implementation deadline of July 2000. Other states based 
their ETP list on rosters of vendors that had been approved by state certification bodies. 





For example, in Maryland by state law any entity in the state that offers training services 
that is open to the general public must be certified by the Maryland Higher Education 
Commission (MHEC). The state felt that this list served as a very natural starting point 
on which to base the ETP list. 

Disseminating Information about Vendors 

The Workforce Investment Act (Section 122e4A) stipulates that the ETP list must 
be distributed to the One Stop delivery systems within the State and “be made widely 
available to participants in employment and training activities.” Moreover, the ETP list 
should be bundled with other sources of information in a consumer reports system that 
would, according to the WIA regulations, help adults or dislocated workers to “fully 
understand the options available to him or her in choosing a program of training services” 
(20 CFR 663.570). 

As a subsequent section of this chapter will discuss, performance information on 
vendors was almost completely absent at the time of our site visits. Thus, the information 
that states could provide for the ETP list consisted for the most part of basic information 
about vendors (e.g., name, address, etc.) and their programs (course name, duration, 
costs). With wide distribution as their main goal, states made this information available 
either electronically, in paper form, or through a combination of both methods. On the 
one hand, a few states have elaborate systems that enable users to search electronically by 
geography and key word, “shopping cart” features so that users can easily compare across 
vendors, links to each vendor’s Web home page, and other features. A few states (e.g., 
Missouri, North Carolina, and Pennsylvania) have also developed (or are planning to 
develop) customer feedback systems that enable those who have undertaken training to 
post comments on the program’s quality on the Web. At the other extreme, some states 
thus far (or at least at the time of the site visit) have only hard-copy listings or, at best, 
electronic ETP lists in the form of Microsoft Word files or spreadsheets. Clearly, in 
many states much developmental work on the CRS infrastructure remains to be 
accomplished. 

Apart from the sheer amount of work that yet needs to occur, states are grappling 
with issues regarding how the data — or how much of the data — should be displayed. For 
example, respondents in some states voiced concern that users, including customers of 
the One-Stop center and WIA partner programs, might misinterpret performance 
information about vendors or not understand the requirements for eligibility. One state 
was contemplating handling this issue by suppressing much performance information, 
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and thus might simply indicate (in an ETP list that included non-lTA eligible providers) 
whether the program was W1A approved. 

Another consideration concerned how or whether to link information about vendors 
with other data sources in a comprehensive management information system. A few 
states, generally those that have created ETP lists that address the needs of many different 
audiences, have been inclined to link these different data systems. In use by customers, 
case managers, local boards and multiple partner programs, they reach a large, diverse 
audience. Features of these systems vary but can include customer tracking, performance 
information, and reporting capabilities. For example, Georgia has decided to put its ETP 
list and CRS on the state-run Career Information System used by multiple program 
partners in its One-Stop centers. For this state, the existing statewide electronic system 
serves as the best distribution mechanism available and ensures that a large audience will 
be reached. Placing the list on this existing system also allows the state to take full 
advantage of previous state efforts as well as funding earmarked towards electronically 
linking the state. 

Other states also have future plans to link the ETP list to the larger MIS. However, 
our respondents pointed out to us that these efforts will require much time and additional 
resources beyond the scope of this grant in order to be realized. It will be interesting to 
see in the next round of visits to these demonstration sites how much progress has been 
realized and how much, if any, additional funding was invested in the effort. 

Still other states embed their vendor list on a larger career-oriented web site, with 
diverse resources and tools to assist customers in making training decisions. These other 
resources might include information about transportation and childcare services, sources 
of financial aid, career planning tools, assessment instruments, and labor market 
information, among other things. An example of one such system, Maryland’s 
CareerNet, is described below: 

Maryland’s CareerNet directs customers with the click of a mouse from 
information about vendors to a variety of other tools and resources that might be 
useful to those making training decisions. For example, customers can take various 
on-line personality tests, a work- interest quiz, and a career inventory. Another link 
directs one to labor market information and information about applying for 
financial aid. Still another provides tools for the job search, including how to 
prepare a resume and develop a job search strategy. 

By including the list in this broader context, the state and local areas are able to 
emphasize the overall vision oftraining as a path to self-sufficiency and career growth. 
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In any case, the time and resources needed to create even less ambitious consumer 
report systems are enormous, as the study sites are finding. Many of these demonstration 
states, even those farthest along in the process, had not fully developed these systems and 
were a long way from having the performance information that WIA will eventually 
require for ITA-approved vendors. In fact, faced with projected substantial time lags 
before some state systems are fully developed, some local-area grantees are using local 
resources to develop interim solutions. For example, both Southwest Connecticut and 
Metro Portland realize that it will be a long while before their states’ consumer reports 
systems are fully functioning and for this reason are contemplating developing local-area 
versions for their own use. 

Initial Eligibility Application Procedures and Requirements 

Developing the vendor application process was an important first step in building 
the ETP list. The two key components of this process were marketing to eliciting 
applications from vendors and developing the actual application procedures. 

Marketing Efforts by States and Local Areas 

States and local sites varied in their approach to bringing vendors to the eligible 
provider list. The approach they adopted was influenced to some degree by the 
fundamental design decision at the state level of whether the consumer report system was 
intended to include all training resources in the state or was to be limited only to a 
narrower subset. In the former approach, states and local areas needed to solicit all 
possible vendors and then select a subset that met qualifications of the eligible provider 
list for ITA purposes. These sites felt that the first order of business should be to develop 
an inclusive list of ITA-approved vendors for initial eligibility — a Yellow Pages, as one 
respondent called it — and then worry about developing performance criteria to winnow 
the list for subsequent eligibility. Georgia is a good example of a state trying to expand 
its list, especially in its rural areas, where customers might have few choices without 
special outreach efforts. 

The state is very active in attempting to market the eligible provider list to 
vendors. In addition to getting the WIBs to develop active solicitation 
programs, it is working to identify vendors who might be able to serve rural 
areas, in which customers might otherwise encounter severely constrained 
choices. The state also provides active technical assistance to vendors to 
adjust their programs to help them meet training needs in particular areas. 

Other states take a more limited approach. Under this scenario, these sites could 
confine marketing to ensuring that the list is large enough to provide sufficient choice for 
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their ITA customers. In a minimalist approach, local areas needed to offer just enough 
marketing assistance to bring in their good JTPA vendors. For example, at least one state 
indicated that it wanted to get its consumer report system operating properly with a 
smaller universe of eligible providers before moving to expand the list to encompass all 
training providers. It did not expect to push for this expansion until it addressed 
performance issues under subsequent eligibility in PY 2001. 

With respect to marketing and disseminating applications to vendors, both states 
and local areas played important roles, but the relative balance between the two varied 
from state to state. Some states have taken a very proactive role in the application 
process. These states sent applications or information on the ETP application process to 
all state licensed or certified training vendors and encouraged them to apply. These states 
felt that it was important to cast as wide a net as possible and make the application 
process as easy as they could for vendors and local areas. Adopting this strategy also 
helped them alleviate their concern that vendors might otherwise be unaware of the new 
ITA system and accompanying application procedures; by sending an application to all 
training providers on a master list, the state could thus ensure that all vendors would have 
access to the application materials. For example, Maryland identified all vendors in the 
state that were certified by the Maryland Higher Education Commission and who were 
thereby authorized to market training services in the state. Local areas were then 
encouraged to follow up, to encourage those vendors to submit an application for ETP 
eligibility. 

By contrast, some states left it almost entirely up to the local areas to notify 
vendors of the procedures for applying for ITA eligibility, or they otherwise encouraged 
local areas to engage in heavy recruitment and outreach. For example, Southwest 
Connecticut, SELACO, and Cincinnati had periodic meetings with vendors in their areas 
to explain the ITA system and the application procedures, to complement informational 
sessions conducted by the states. Cincinnati was notably aggressive in its marketing 
efforts, holding quarterly meetings with vendors that will continue beyond initial 
eligibility. 

Submitting an Application for Eligibility 

All of the states in this demonstration followed a fairly standard model based on the 
legislative requirements for the initial eligibility process, which stipulate that vendors 
should submit an application to the local board for the area in which the provider desires 
to provide training services. All states followed this requirement, but some variations did 
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occur due to basic administrative differences. The typical model used by states aligns 
with the procedures required in the legislation and can be described as follows: 

1 . The vendor completes an application to the ETP list for each of its 
programs. 

2. The application is submitted to the local area board either directly to the 
local board where the vendor is seeking eligibility, or to the state who in 
turn forwards it to the relevant local areas. 

3. The local board reviews the application and forwards it to the state with a 
recommendation for entry on the ETP list. 

4. The state reviews the application and either denies or accepts the 
application for eligibility. 

5. The provider and local board receive notification of acceptance or denial. 

6. Providers who are denied entry to the list may choose to appeal to the 
state or local board following set appeal procedures. 

The one variant, implicit in the description of step 2 provided above, is that in most 
states vendors would submit applications directly to the local area or areas to which they 
were applying, while, in other states applications would be sent to the state who would in 
turn forward them to the relevant local area or areas. The advantages of the latter 
approach, it was felt, were that vendors need not submit multiple applications for the 
same program to different local areas, potentially needing to meet various and 
idiosyncratic local area guidelines, but could instead submit a single application only. 

This more streamlined process was viewed as removing a potential major impediment to 
vendors’ wanting to apply, as well as substantially easing the burden on the local areas. 

In either case, the local board remained the first arbiter of whether the vendor’s 
application was approved. Thus, even in the states that were recipients of applications 
first, the local board still made the initial decision about whether the vendor’s application 
should be recommended for inclusion on the state list. For example, in Michigan, the 
state receives the application and then forwards it to the appropriate local boards for 
approval. The local boards in this example still retain the authority to give, or withhold, 
their approval. 

Also as a way of streamlining the application process, over half of the states we 
visited made it possible for vendors to submit applications electronically. The remaining 
states that do not yet have electronic application procedures are creating the infrastructure 
necessary for them to do so, in some cases using funds from this grant. Although the 
specifics vary, where electronic application is an option vendors are able to download a 
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copy of the application from the Internet to fill out and mail in to the appropriate local 
board or state agency. Most of these states also allow vendors to email or submit a 
completed application over the Internet directly to a board or state clearinghouse. Several 
of the states are allowing both paper and electronic applications as acceptable forms of 
submission, though at least one of the states will only allow electronically submitted 
applications once its electronic system is fully functioning. 

States that accepted electronic applications felt that doing so expedited the process 
in a major way. For example, the burden of having to keypunch data entries could be 
entirely eliminated. Additionally, a preset electronic application ensures that all 
necessary information will be collected from the vendor, obviating the need for call- 
backs or otherwise dealing with incomplete applications. For example, in Texas, the 
electronic system will not allow an application to be submitted if certain data fields are 
incomplete or missing. With an electronic process in place, approval of applications also 
proved to be a much easier task, in that a computer program can be created to check 
automatically that vendors are meeting minimum requirements for approval to the ETP 
list (such as performance benchmarks). This benefit will be most noticed in subsequent 
eligibility as vendors and states are faced with processing the large amount of anticipated 
performance data. These new electronic systems should make the application process 
and the submittal of performance data easier for vendors, local boards, and the state alike. 

Information Required for Application 

All of the states require the same basic vendor and program information on the 
application form. This basic information includes vendor/institution name, contact 
person, address, information on accreditation, and training provided. Typical program 
data include name of program, course content, certificate/degree awarded, mode of 
delivery, hours of instruction, credits, costs, and program goals. As we will discuss 
shortly, most states waived any requirements for performance information for initial 
eligibility, even for non-exempt vendors, though it could be provided if the vendor had it 
available. For example, in California, the state invited vendors to submit recent 
performance data but flagged this information as “self-reported by the providers and has 
not been verified.” This performance information does not effect a vendor’s inclusion on 
the ETP list. 

Several of the demonstration states also explicitly required that providers have 
obtained state certification and licenses to provide training before they apply for 
eligibility to the ETP list. These certifications and licensures were mandated by state law 
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well before the enactment of W1A. States gave two reasons for requiring this basic state 
certification or license before an application for ETP eligibility would be approved. First, 
they used the state certification as a way of ensuring that vendors met minimum state 
standards for quality. In addition, reliance on a pre-existing list of state-certified vendors 
was very expedient, in that it provided an obvious starting point for developing the ETP 
list and served as a ready means for identifying vendors from whom applications should 
be sought. 

These certification processes vary in scope and content to some degree. In general, 
however, they focus on process rather than outcomes, considering, for example, whether 
curricula and equipment are up-to-date, whether facilities are adequate, and whether 
student-to-staff ratios are low enough. In Maryland, the state requires the Higher 
Education Commission (MHEC) to approve the provider in a fairly rigorous site visit. 
Every public or proprietary post- secondary school offering training to the general public 
must have certification from MHEC, so it seemed obvious to use this list as the starting 
point for 1TA eligibility. Similarly, in Oregon respondents stated that requiring that 
providers be licensed helps create an initial quality assurance before ETP subsequent 
eligibility rules are put in place. 

One complication is that states requiring state certification for providers have in 
some cases had to set up special processes to deal with the large influx of applications 
from vendors applying for inclusion to the ETP list. In Oregon, for example, while 
reviewing applications to the ETP list, state representatives discovered that some 
providers were offering training courses without proper certification. To ensure that 
applications from new providers and those not certified will be sped through the 
certification process, the state has set up a quick certification process so that providers 
can get on the ETP list in a timely way. Though the state does not want to create too 
many bureaucratic processes, they feel that this state certification is necessary and 
ultimately a benefit to all parties. 

An additional difficulty is that, in some cases, requiring state certification is serving 
to exclude from the ETP list vendors that might have been used frequently under JTPA, 
sometimes with good results. For example, in Connecticut, providers without 
certification from the Department of Human Services cannot offer training services to the 
public or advertise. However, under JTPA, non-certified vendors in this state could forge 
agreements with a local job-training agency to conduct class-size training. Given WIA’s 
reduced emphasis on contracted training, providers lacking certification will be little used 



and might have difficulty finding enough other business to stay afloat. Now, some of 
these training providers are seeking state certification to stay afloat. 

Developing Regional Approval Systems 

As we have discussed, Local Boards review vendors’ applications for eligibility 
and decide whether or not they should be approved and forwarded to the state for 
inclusion on the state list. Local areas also are empowered to set more stringent 
performance criteria beyond those established by the state. At the same time, pursuant to 
20 CFR 663.585, some sites hold that 1TA holders are free to choose any vendor on the 
state list, so long as the training program is for an occupation in demand. This set of 
circumstances gives rise to the possibility that a customer issued an ITA by one local area 
could use it to procure training services from a vendor that had been denied eligibility by 
that local area but approved by an adjacent one. 

Several of the case-study sites were disturbed by this possibility, and stated that 
they would be very reluctant to expend their training funds for services at a vendor that 
they had denied for eligibility. As a precaution, they were attempting to coordinate the 
vendor approval process across adjacent local areas, at least to some degree. Doing so 
was in fact a chief focus of Macomb- St. Clair’s grant plan. This grantee had used its 
grant funds to hire a consultant who was serving as a regional coordinator responsible for 
identifying existing variation across local areas in the Detroit metropolitan area, not only 
with respect to their vendor approval criteria, but also regarding caps on ITA costs, the 
maximum allowable duration of training, how demand occupations were defined, and so 
on. The consultant is facilitating the coordination of policy formulations across the local 
areas to achieve greater consistency. 

The several local areas that make up metropolitan Atlanta developed a different 
approach, specifically focused on the problem of ensuring regional agreement with 
respect to vendor suitability. These areas empowered the Atlanta Regional Commission 
to act on their behalf in reviewing vendor applications and deciding which ones should be 
forwarded to the state for approval. 

A third demonstration site, North Carolina, adopted yet a different approach to the 
same problem. This state was including separate data fields in its consumer report 
system to identify specifically which programs had been approved by which local areas. 
With this information readily at hand, potential trainees and their case managers could at 
least quickly identify vendors that the local area had deemed as unsatisfactory. 
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Subsequent Eligibility Applications and Procedures 

Only Texas, one of three W1A early implementing states in our sample, had 
actually begun subsequent eligibility at the time of the site visit. All the others did not 
face a deadline for procedures and standards until at least PY 2001 . Thus, few of them 
had finalized their subsequent eligibility policies and, in some cases, had given them little 
thought. Other than Texas, only Maryland, Missouri, and Ohio had made considerable 
progress in developing subsequent eligibility requirements. In the remaining states, state 
representatives and 1TA work groups were hard at work. Several of the states hoped to 
have recommendations and decisions from these work groups during the first few months 
of 2001. 

One of the reasons for this delay was the difficulty in establishing performance 
guidelines as well as gathering and monitoring performance data. Determining final 
performance measures has been difficult as states attempt to satisfy all entities involved, 
including Local Boards, community colleges, private training vendors, One-Stop center 
operators, case managers, and the customers themselves. As described in the next 
section, states have had to change existing policies and create new definitions before 
determining final procedures. 

Given these difficulties, several of the states admitted to purposefully postponing 
subsequent eligibility for as long as possible. In fact, one state anticipates requesting an 
extension to use initial eligibility guidelines for a full two years. 

Most of the states expect a huge drop-off in the number of providers on their ETP 
list once subsequent eligibility begins. Texas, the only state that has entered subsequent 
eligibility, in fact experienced a substantial drop-off of vendors applying and becoming 
eligible for the ETP list. One respondent estimated that the ETP list had fallen by 85% 
from initial to subsequent eligibility. The overwhelming reason given for this drop-off 
was that vendors, mostly community colleges, were unwilling to undertake the difficulty 
and expenditure of time and resources necessary to gather and submit performance 
information, when they expect that only a trickle of 1TA customers will result. 

Another issue that Texas has dealt with concerns the process for approving newer 
programs and those that have not served any W1A participants in a program year. With 
DOL’s approval, the state decided that such programs may be re-certified for one 
additional year (this option does not apply to performance data pertaining to the WIA 
population because the vendor may collect this information from the state directly). If a 
vendor has not trained any WIA participants 90 days prior to the expiration date for 



initial eligibility, then a local board may also exercise this option and grant the vendor re- 
certification for one additional year. 

Performance M easures 

Developing performance benchmarks forjudging vendors’ success at serving 
trainees, as is envisioned by the ETP application process, has proven to be extraordinarily 
difficult. As a stopgap approach, most sites in our study waived performance 
requirements for initial eligibility and have yet to make much progress in developing 
them for subsequent eligibility. Among the thorny issues with which they are grappling 
are defining key terms and developing necessary and appropriate data collection systems. 

Performance Criteria for Initial Eligibility 

In keeping with WlA’s requirements, degree- granting post- secondary institutions 
are granted automatic initial eligibility so long as they apply, but other training vendors 
may need to meet minimum performance requirements. In fact, all but two of the states 
we visited as part of this demonstration, Missouri and Texas, waived any performance 
requirements for initial eligibility for all vendors. This decision greatly affects the way 
initial eligibility functions in these states as well as how subsequent eligibility will work 
in the next program year. The overwhelming reason that states gave for waiving 
performance requirements for initial eligibility was that they were not sure what 
requirements they should adopt and needed to confront complicated issues about how key 
measures should be defined and operationalized. Moreover, in most of these states the 
information systems needed to collect, process, and display the volume of information 
that performance requirements would imply were not operational at the time initial 
eligibility began in July 2000. Clearly, states expected that establishing performance 
requirements for eligibility would give rise to extraordinarily complicated measurement 
and data issues, and they wanted to give themselves time to address them in a thoughtful 
and deliberate way. 

Another reason for states’ waiving performance measures under initial eligibility 
was to address the concerns of many vendors, especially public institutions, who were 
anxious about what might be required and how the new system would work. As will be 
discussed in the next chapter, community colleges admitted that they were not nearly 
ready to provide performance information about their programs and were very 
uncomfortable with the levels of performance they might show if they did so. For 
example, some community colleges worried that their open- admissions policies would 
make them appear to be very poor performers. In recognition of these very legitimate 
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concerns, states made the decision to waive performance requirements as a sign to 
vendors that they are important to the system and that the state understands their 
concerns. 

States also felt that waiving performance requirements would help to ease the 
transition to W1A for vendors. Providers, like local areas, case managers and customers, 
need time to get used to the new way of conducting business that an 1TA and ETP system 
implies— for example, by taking more care to support students through training and 
afterwards to see that they complete their training and obtain employment. Moreover, on 
a more practical note, waiving performance measures allowed the states to retain the 
large numbers of providers they felt they would lose if performance requirements were 
imposed. During initial eligibility, states hoped, they would be able to work out many of 
the system difficulties and have the opportunity to demonstrate to providers that ETP 
eligibility was valuable and worth their added effort. Interestingly, many of them also are 
requiring vendors as part of the application for initial eligibility to agree to begin 
gathering and submit the required performance data in subsequent years. Thus, they are 
attempting to ease the burden on vendors in the short-run, while endeavoring to get them 
to commit to be players in the system once performance requirements are imposed. 

Performance Criteria for Subsequent Eligibility 

Approximately two-thirds of the states in our sample were still grappling with these 
issues at the time of our site visit, and thus had not decided on final performance 
measures for subsequent eligibility. Many of them, in fact, were not even close to doing 
so. Conversely, four states finalized their performance measurement policies, or at least 
had developed concrete recommendations that were being fine-tuned or circulated for 
comment. 

Across all states, however, determining the performance criteria for subsequent 
eligibility as well as developing the necessary data systems to support them have been 
time consuming exercises, as states have endeavored to make decisions regarding 
complex issues while balancing the needs of all parties involved. The majority of the 
states convened work groups made up of representatives from local boards, proprietary 
schools, and community colleges and state officials. In one of the states the work group 
met every other week for six consecutive months to determine what performance data 
vendors should report for subsequent eligibility and what minimal levels of performance 
would be required. 
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A key consideration has been establishing performance criteria that are rigorous 
enough to ensure a minimum level of quality among vendors who are certified as being 
eligible, while not establishing them so high that so many vendors will be excluded from 
the ETP list as to unduly restrict customer choice. Establishing fairly lenient criteria for 
eligibility also brings the associated risk of reduced expectations by conveying the 
message that mediocre performance is satisfactory. Another very practical consideration 
has been knowing at what levels to establish performance requirements in the absence of 
having good historical data about what actual performance has been. For example, states 
that in principle would like to establish performance levels at a certain percentile of 
performance find that they have no way of knowing what absolute score that percentile 
would correspond to. 

The four states that were close to finalizing their performance requirements at the 
time of our site visit show that states balance these objectives and considerations very 
differently, partly as a function of the sophistication of their existing data systems and 
their previous experience with collecting performance information from vendors. For 
example, Maryland has limited historical information about vendor performance and is 
thus leaning towards establishing fairly relaxed performance criteria for subsequent 
eligibility for at least the next year or two, until enough performance data are 
accumulated so that it would be positioned to make a more informed decision. In this 
state, vendors, in collaboration with the state, will be collecting performance information 
on all the WlA-required measures, but, as of now, the only performance benchmarks 
required for subsequent eligibility will pertain to employment rates for WLA- funded and 
all trainees. The exact minimum benchmarks on these two employment measures were 
still being debated. However, the state will endeavor to establish a level that will not 
exclude an excessive number of vendors, while being high enough to represent a sound 
achievement that will keep vendors competitive. In setting the benchmarks, the state will 
also take into consideration performance levels that the state has negotiated with the DOL 
Regional Office on the related W1A core measures, the adult and dislocated worker 
entered employment rates. No other performance requirements will be imposed at this 
time. However, the state envisions using the first several years of subsequent eligibility 
to begin amassing a database of vendor performance on all remaining measures, so that 
benchmarks can be established on them in a reasonable way later. 

At the other extreme, Texas, with a longer history of tracking the performance of 
training providers, has decided on a somewhat more stringent approach for subsequent 



eligibility. This state has already decided on its minimum benchmarks for PY 01 as 
follows: 



© Performance requirements for all trainees 



- Program completion rate: Percentage of all participants 

completing the applicable program conducted by the provider. 


- 60% 


- Entered employment rate: Percentage of all participants who 
obtained unsubsidized employment. 


- 60% 


- Average wage at placement: Average wage at placement 
of all individuals who were employed (set at 1 20% of 
federal minimum wage). 


- $6.18 


® Performance requirements for WI A- funded trainees 




- Entered employment rate: Percentage of WI A- supported 
program completers who obtained unsubsidized employment. 


- 65% 


- Employment retention rate: Percentage of WI A- supported 
completers who were employed six months after completion. 


- 70% 


- Average quarterly wages: Average quarterly wage of WIA- 
supported completers who were employed six months 
after completion. 


- $2,512 


Defining Key Terms 





Before establishing performance criteria, states are finding that one important 
conceptual hurdle is deciding how to define key terms. The most fundamental decision is 
what counts as a “program.” Because the ETP list is in actuality a list of training 
programs, rather than just a list of vendors, defining what constitutes a program is 
essential even before the ETP list can be established, let alone before performance criteria 
are imposed. The regulations (20 CFR 663.508) provide some guidance by defining a 
program as one or more courses that, upon successful completion, lead to a certificate or 
degree, a competency or skill recognized by employers, or additional skills generally 
recognized by employers. Certificate or degree programs can be identified as “programs” 
according to this definition very straightforwardly. However, substantial ambiguity 
arises for individuals who may take variable sequences of courses, as they often do at 
community colleges. Although the specific approaches vary, states generally handle this 
dilemma by allowing vendors to self-defme what constitutes a program for which they 
want to seek eligibility. Vendors, in turn, specify the course sequence and completion 
requirement 5 ; as part of the application packet. Also, the same course sequence provided 
by the vendor at different physical locations will generally count as separate programs. 
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Taking this complication one step further, a state-organized workgroup in Oregon 
found that developing a clear definition of a training program required that they first 
clarify what they considered to be an intensive rather than a training service. This group 
created new service categories within intensive service to represent a number of short- 
term training options, including courses in computer literacy, professional development 
skills, and individual employment skills, that involve less than 600 hours of classroom 
time. Because these courses are classified within intensive services, customers can enroll 
in them without having been issued an ITA. Although the state does not require vendors 
to place these courses on the state ETP list, it does recommend that vendors do so for the 
sake of quality assurance. 

States are grappling with a variety of other definitional issues that will also 
determine how vendors’ performance is measured. At the time of our site visit, few 
states had resolved these issues, but most were well aware of their implications. Some of 
these discussions concern how to define an enrollee. Along these lines, one concern that 
vendors have, particularly community colleges, is that some persons might enroll in the 
first course of a sequence that makes up a “program” on the ETP list without necessarily 
intending to complete the entire sequence. Others (e.g., retirees or those pursuing an 
avocation) might complete the program but have no intention of obtaining employment in 
a related field. Under such circumstances, vendors worry that their completion and 
employment rates might look quite dismal if such persons are counted as enrollees, 
without it necessarily reflecting anything meaningful about the vendor’s performance. At 
least a few states have decided to resolve this difficulty by counting individuals as 
enrolled only if at the outset they formally declare their intention to take all courses in the 
sequence and pursue employment at the training’s conclusion. Maryland provides an 
example of this. 

Once the program has been defined and the cohort of those enrolled has been 
specified it would seem straightforward to identify completers. When to count a person 
as a non-completer is proving somewhat more controversial, however, because some 
states want to make allowances for the fact that some enrollees may interrupt their 
training for family or medical, while still intending to complete eventually. One state in 
our sample is fairly hard-nosed in its definition and identifies non-completers as those 
who failed to complete during the reporting period if they were expected to do so. 

Another state measures completion at one-and-a-half times the normal expected 



completion date, to allow for the possibility that the student experienced a temporary 
interruption of training. 

Data Collection 

Because so many of the decisions about how to define performance were still being 
debated in the case-study states at the time of our site visits, little information was 
available about data collection strategies. Nonetheless, we can make some general 
observations based on the progress that states have made or plans they had developed. 

Not unexpectedly, we learned that virtually all of the states are developing 
procedures to use UI wage records to collect employment-related performance data, as is 
required by the legislation. Only one state, Nebraska, expressed a serious concern about 
confidentiality issues that might preclude this possibility, but even it was confident that 
they would be resolved satisfactorily. States cite multiple reasons for relying on UI wage 
matching so heavily. These include the imperatives of ensuring that data are collected 
from a reliable source and in a uniform manner across all vendors. Moreover, using this 
source substantially relieves the burden on vendors, which is a key consideration if one is 
concerned about inducing them to apply for eligibility. In short, the state has the 
resources and technology to collect employment information very readily, whereas many 
providers do not. 

At the same time, the drawbacks of using UI data were recognized very clearly by 
our state respondents. These include the long time lag between when an individual 
begins employment and when the associated employment information is available from 
the appropriate state agency. Incomplete coverage was also mentioned, including the fact 
that some types of employment will not show up in UI files (e.g., self-employment), nor 
will employment in another state (without an inter-state agreement, along the lines of the 
Wage Record Interchange System ). 1 For these reasons, two states have already decided 
that vendors could supplement UI data on employment with information that the vendors 
gather from their own sources (e.g., surveys), if they choose. Other states are still 
weighing this possibility. 

By contrast, data on program completion rates will typically be provided by the 
vendors themselves. In an exception, one state is planning on receiving individual- level 



1 The Wage Record Interchange System has been created to allow states to conduct matches with 
other states’ UI files, but participation in this system is voluntary. 



enrollment data Files from the vendors, from which it will compute each program’s 
completion rate. It believes that this strategy not only eases the burden on vendors, but is 
the best way of ensuring data quality and comparability. We emphasize that, with a few 
exceptions, these decisions were tentative, as states are still formalizing their intended 
procedures. 

Because of the very vocal concerns of vendors about the problems and hardships of 
collecting performance data, a few states have exercised creativity in an attempt to ease 
the burden of collecting performance information for their providers as best they could. 
For example, Pennsylvania, along with a few other states, has adopted a hardship waiver 
for its providers who can prove that collecting performance data will cause a major 
burden to the training organization. Georgia hoped to use performance information from 
accrediting bodies in the state as an alternate means of collecting performance 
information for the CRS, although state officials are finding that most accrediting bodies 
are not very confident of the reliability of their own data and are in fact looking to the 
state’s CRS to meet their own informational needs. 

Another state, Texas, also exercised resourcefulness to ease the burden of 
collecting performance information for their providers. One strategy that this state 
developed was to enable providers who are approved by the Texas Higher Education 
Coordinating Board to use data on program completion rates required for certification 
from that body in lieu of providing data explicitly to measure completion in accordance 
with the CRS, on the grounds that the former is substantially similar. 

Payment Systems 

Because many of the sites used voucher- like systems in the past, there is relatively 
little change in the payment systems that they used. For example, the Pittsburgh, 
Allegheny County, and the Atlanta Regional local areas used the identical methods of 
contracting and paying vendors that they had used under JTPA. In all those sites, the 
financial offices of the LWIAs executed non- financial agreements with vendors on the 
eligible provider list. When a participant receives an ITA, he or she presents the voucher 
to the training vendor, who then submits an invoice to the LWIA’s financial office to be 
paid. 

Carrying over payment methods from JTPA eased the transition to the ITA system, 
but some administrators noted that these “legacy” methods are cumbersome and ill suited 
to the requirements of a market-based system. Some of the sites indicated in their grant 
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applications that they wanted to modernize their payment and financial tracking. Some 
planned to add financial components to their 1TA computer systems to track expenditures 
and provide for payment to vendors. Others wanted to develop systems that would allow 
customers to track their ITA balances electronically. However, for the most part these 
financial systems were still under development at the time of the site visits. 

In another innovation, one site, the Three Rivers Local Board, planned to introduce 
a “smart card” payment system, which would not require the LW1A staff to prepare 
warrants and issue paper checks. It planned to use its funds from the demonstration grant 
to work with a local bank to develop such a smart card. Board administrators hoped that 
this would free them from the strictures and costs of the cumbersome public financial 
system. Unfortunately, the board had to drop the idea because it proved too expensive 
for small-scale ITA operations. 

However payments are made, they can be tied to performance benchmarks realized 
by the trainee. Only one site, Southwest Connecticut, was exercising this option. In this 
site, the local area would pay 50% of the tuition to its vendors only when the participant 
completed 50% of the program, an additional 25% when the participant completed the 
training, and the final 25% when the participant obtained employment. 

Summary 

The development of the ETP list and consumer report system is extraordinarily 
complex. Not only do they require an enormous investment on the part of local areas, 
state administrators, and, to some degree vendors, but thorny conceptual issues need to be 
resolved, relating to what the CRS should represent, what benchmarks should be used to 
establish eligibility for vendors, and how performance measures should be defined and 
measured, among other issues. With few exceptions, the states and local areas in our 
sample had made limited progress in resolving these issues. 

Decisions or policies that had been established reflected substantial variability 
across sites, as each endeavored to shape the ETP and consumer report system in a way 
that it felt would best meet the needs of the diverse stakeholders involved, including 
program administrators, trainees, and public and private vendors. For example, some 
states were developing a CRS that included non-lTA approved vendors, in an effort to 
improve the comprehensiveness of the system for customers in core services and leverage 
the resources they were spending on system development; others, by contrast, were 
developing a narrow list that included just eligible training providers under W1A. 
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Similarly, some were casting a wide net by endeavoring to get as many vendors initially 
approved to accept the ITA as possible, with the thought that they would winnow the list 
in the future once data systems for measuring performance were better established; 
others, by contrast, were more tentative starting out and felt that their interests would be 
better served by starting with a smaller list of vendors whose quality they could attest to, 
with the plan for expanding the list once the basic infrastructure was better established. 

At this stage of system development, states also varied greatly in the complexity of the 
computer interface that they offered customers and the comprehensiveness of the 
information and tools that were available on-line. 

Although all states need to make substantial progress to attain their complete vision 
for system development, no doubt the biggest stumbling block that all of them have 
realized thus far has been with respect to how the performance of vendors should be 
measured. Conceptual issues having to do with the definition of key terms — such as who 
counts as an enrollee or completer— are still being resolved in all but a few of the states. 
Operational issues are just as complex, such as establishing the data interchange systems 
for UI matching. 

Clearly, substantial work remains to be accomplished if WIA’s vision for an ITA 
training system with rich information to support customer choice is to be realized. 
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V. TRAINING VENDORS: 
INVOLVEMENT AND PERSPECTIVES 



This chapter discusses the 1TA system from the point of view of the institutions 
that provide training to ITA holders. Training vendors are a diverse group. We use this 
diversity as a departure point to view the network of vendors available to any particular 
local demonstration site and examine how that network serves the needs of the ITA 
customers and system. Our discussion of vendor interactions with customers focuses on 
several key factors affecting customer choice, including the number of vendors and 
programs, vendor accommodations to student needs, and recruiting practices. We 
conclude the chapter with a general assessment of the impact of ITAs on the number and 
types of vendors and on the general structure of the training marketplace. 

Types and N umbers of Vendors 

In the local areas we studied a wide variety of vendors are present, running the 
gamut from four-year colleges to proprietary vocational schools to community-based 
organizations. They are quite distinct in the way they relate to the ITA system and the 
training market niches they fill. We looked primarily at public and proprietary schools in 
this first round of site visits, and found important distinctions between them. 

The public institutions, especially community colleges, are ubiquitous and 
represent an important training resource in all the sites. Community colleges are notable 
for their lower tuition costs, breadth of offerings, and access to degree programs. 
However, the colleges tend to adhere to a regular academic calendar, with courses 
beginning at only a few regularly scheduled dates. Moreover, because community 
colleges are typically very large institutions, non-traditional students can easily feel lost 
in this setting. 

By contrast, the proprietary schools whose representatives we met characterized 
themselves as being active in the marketplace and highly responsive to customer needs. 
For example, a vocational school that trains people in a number of manual trades in 
Pittsburgh and Allegheny County offers shorter term evening training in some subjects 
for currently employed workers. This school offers an associates degree in many of its 
programs, and, like other proprietary schools that we studied, has linkages to degree- 
granting four-year colleges. The school also would esta„. a class whenever a public 
agency could bring together 15 students. 
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Some proprietary schools, especially those in large metropolitan areas with a large 
variety of schools, specialize in serving particular groups or providing particular learning 
environments. In the Atlanta Regional local areas, for example, where there is a strong 
emphasis on computer training, some schools specialize in serving dislocated workers 
while another serves low-income adults with limited basic skills. The latter school offers 
a “boot camp” approach through which instructors helped students with a range of basic, 
life, and labor market skills in addition to teaching them occupational skills. Other 
schools use different curricula and pedagogical methods designed to appeal to different 
types of students. For example, one computer school uses an intensive approach in the 
classroom and expects students to complete practice exercises on their own in the 
computer labs. Others offer a more structured classroom environment with greater 
reliance on the instructor demonstrating techniques to solve relevant computer and 
network problems. 

While we did not visit with any community-based organizations that provide ITA 
training, Local Board and One-Stop staff noted that many of these organizations provided 
training to economically disadvantaged adults under JTPA, but are not likely to be 
successful under an ITA system because of their traditionally narrow customer base. In 
addition, their frequent reliance on contract training puts them at risk of losing more 
customers than they might gain in a competitive marketplace. Further, they are usually 
thinly capitalized and are likely to have difficulty coping with an irregular flow of ITA 
students. 

Not only types but also sheer numbers of vendors are important in an ITA system, 
because a larger number of vendors and programs obviously contributes to enhancing 
customer choice. Because the number of training vendors generally is larger in the 
demonstration sites located in large metropolitan areas, customers in these areas will 
generally have a greater degree of choice. For example, at the time of the site visits 
Baltimore reported 70-80 vendors, each with about 5 or 6 programs. Macomb-St. Clair 
and Indianapolis had 349 and 240 programs respectively. By contrast, Nebraska had only 
100 vendors statewide. Not surprisingly, our urban respondents felt that they had a good 
network from which customers could choose. Yet, even our respondents in rural areas 
and smaller urban areas considered their vendor lists to be sufficient to provide ITA 
customers with good quality training even though there were fewer choices. 

Unfortunately, the range of choice may decline when subsequent eligibility begins, 
regardless of the area’s population density. Texas had 8,000 programs statewide for 
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initial eligibility, but that number fell to 1,000 for subsequent eligibility. Moreover, the 
number may fall even further if more of the state’s community colleges decide not to 
participate after the first year of subsequent eligibility, as they have threatened to do 
because of their objections to the performance tracking requirements of the eligible 
provider list. Staff in the Capital Area (Raleigh), which has over 200 vendors with a 
wide range of programs, also expect their vendor list to contract substantially once full 
performance information is required. 

Some local areas are trying to counteract these trends by expanding the list of 
vendors through outreach, while others are trying innovative methods to tap new sources. 
The Lancaster County LWIA in Pennsylvania, for example, is using demonstration grant 
funds to increase the scope of its vendor group by linking the ITA to the training 
resources of local employers, as follows: 

This LWIA is working to include training provided by the area’s businesses 
for its own employees as part of the ETP system, with the expectation that the 
training would thereby become accessible to ITA holders not employed by 
the firm. Such an arrangement is believed to be beneficial to employers, as it 
enables them to share their training resources, to some degree defray the costs 
of their own in-house training, and participate as part of a larger training 
marketplace. At the same time, it benefits the WIA system by increasing the 
training choices for ITA and other WIA participants. Thus, Lancaster 
expects that this innovation will create a more efficient local market for 
training. However, it is still working on methods to measure performance for 
the employer-based, in-house training. 1 

Vendors and Customers 

Because they can no longer count on contracts, or even direct referrals, from job 
training, as they may have in the past, vendors must be responsive to customers’ needs 
and, potentially, engage in some direct recruitment of ITA holders if they are to be 
successful training vendors under WIA. 

Vendor Responsiveness to Customers 

Vendors can be responsive to customers in several ways. Most importantly, they 
can offer successful programs that prepare ITA holders for good quality jobs that are in 
demand and work to ensure that their trainees complete programs and get placed at high 
wages. This is in essence the training marketplace that WIA envisions. However, 



1 We have recently learned that this effort may be abandoned, as a new state law is requiring that all 
training vendors must be state certified. 
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vendors may also respond to customers’ needs through improvements in accessibility, 
such as reducing costs or adjusting the timing, location, or duration of programs and 
adding auxiliary services. 

On the topic of costs, our respondents generally did not suggest that vendors are 
responding to customers through pricing mechanisms, even in those sites where training 
costs regularly exceed 1TA caps, and Pell grants and 1TA holders must contribute 
personal funds or take out loans to make up the difference. 

The timing of programs plays an important role in responding to customer needs 
and has great immediacy to prospective trainees. Proprietary schools are widely 
considered to be quite responsive in this regard. Some schools, especially those 
providing computer training, offer open entry so that a student can begin the training at 
almost any point. One such school in the Atlanta Regional local areas, for example, 
accepts students at any time, although it suggests to prospective students that the best 
starting times are on Mondays and Wednesdays. Although it is less common, community 
colleges can also be flexible. A community college in the Macomb-St. Clair area, for 
example, increased the frequency with which it offers its courses, from three times-per- 
year to five times-per-year, to be more responsive to its prospective 1TA students. 

Duration of the training period is another dimension that is important to students. 
Unemployed workers naturally want to reduce the duration of their unemployment. In 
addition, when they are receiving unemployment insurance, trade readjustment 
allowances, or another form of temporary income support, they have a strong incentive to 
conclude their training before the income support expires. Further, in tight labor markets, 
trainees are eager to complete training as soon as possible to take advantage of favorable 
employment conditions. Several respondents noted that proprietary schools often 
provided shorter, more intensive programs than their community college counterparts. 
Nevertheless, we found at least one community college in Metro Portland that is working 
with the One-Stop center to develop shorter-term training to meet the needs of 1TA 
holders who are increasingly interested in shorter-term training so they could become 
employed quickly. 

The wider availability of auxiliary services for 1TA holders, such as counseling, 
study skills workshops, and placement assistance, which we discussed in Chapter 111, is 
another feature that distinguishes proprietary schools from community colleges. The 
proprietary schools commonly deliver such services directly to their students, while to 



take advantage of comparable services in a community college, students typically must 
seek out those services on their own. 

Recruiting Customers 

Some training vendors, especially those who may be heavily dependent on ITA 
customers, are not passive players in the ITA process, simply waiting for customers to 
choose them from the eligible provider list. Instead, they may try to actively market their 
services to customers and staff within the One-Stop centers. 

Perhaps the most common technique that vendors use to recruit potential ITA 
holders is the reverse referral, whereby participants come into the One-Stop center 
seeking an ITA because they were referred there directly by a training vendor. This 
practice was common under JTPA, when most participants were slated to receive 
training. In this model, a vendor recruited an individual who was interested in training, 
through advertising, word-of-mouth, or other marketing techniques. 2 The vendor then 
referred the person to the JTPA program with the expectation that the individual would 
be determined eligible for JTPA and sent back to the school for training. The ITA 
process under W1A introduces considerable uncertainty into this practice. If an 
individual were referred back to a One-Stop center, that person would first have to 
progress through core and intensive services and be determined eligible for training, a 
less likely prospect than under JTPA. Second, the individuals would go through the ITA 
decision-making process, during which they might choose another vendor from the 
eligible provider list. While respondents in one local area think that these uncertainties 
would tend to diminish the use of reverse referral, we found some evidence that the 
practice is continuing. One vendor in the Atlanta Regional local areas, which makes 
reverse referrals extensively, was quite concerned about the risks involved. One of its 
officials wanted to place a staff member at the One-Stop center to help guide their 
reverse-referred people through the process to ensure that the school would not lose them. 

Vendors’ on-site presence at the One-Stop center can occur in other ways as well. 
At least two local sites invite vendors into their One-Stop centers to provide One-Stop 
customers in core and intensive services with labor market information about the 
occupations for which they provide training. In one site, training vendors also provide 



2 In some areas, vendors held outreach contracts from the Service Delivery Areas to find 
participants, determine their eligibility, and enroll them in their own training services. 
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direct services such as career exploration or job search workshops. Some center staff, 
however, find that these workshops are often thinly disguised recruiting efforts and they 
make efforts to ensure that the content remains legitimate. Elsewhere, the Atlanta 
Regional local areas invite vendors to their centers semi-annually to make formal 
presentations to the counseling staff about their services. 

Vendors’ Responses to the ITA System 

Vendors face two major tasks in providing services under the ITA system, applying 
for eligibility and submitting performance data. With respect to the first of these, most of 
the vendors we talked with reported that the application process was fairly easy. While 
they still must fill out multi-page applications and identify all their programs, the absence 
of performance reporting requirements in 1 1 of the 13 states eased some of the 
controversies involved in creating the eligible provider list during initial eligibility. In 
fact, most vendors felt that the application process to get on the initial eligible provider 
list was fair and reasonable, although some found it to be burdensome. 

The burden will become much greater under subsequent eligibility, when 
performance information is required. Reactions of the community colleges to this 
requirement are generally quite different from those of the proprietary schools. We found 
the former group to be, on average, quite distressed with the idea that they would need to 
provide performance data. For example, in SELACO, the local community college did 
not apply for the eligible provider list because of the performance requirement. In Texas, 
the number of programs dropped statewide by about 80% from initial eligibility, when no 
performance information was required, to subsequent eligibility. Nearly all of this drop 
came as a result of the community colleges’ refusing to participate. 

Three types of objections were raised by the community colleges. First, several of 
them indicated that their open admission policies would harm their performance 
compared to proprietary vendors with more selective admission policies. For example, in 
Baltimore, state law requires the Baltimore City Community College to admit all 
applicants regardless of their preparation. Consequently, it experiences a 50% dropout 
rate. Similarly, a community college in Cincinnati also has an open admission policy and 
a high dropout rate. Under these circumstances, these institutions felt, their performance 
results could be highly misleading. 

The second principal argument we heard is that the costs of obtaining the 
performance information required of them by W1A are too high. For example, Indiana’s 
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public postsecondary schools consider ITA customers to be an insignificant part of their 
revenue stream. With a budget of nearly a billion dollars, the schools are not eager to 
spend the resources to meet performance indicator requirements for the relatively few 
customers they might receive through ITAs. 

Finally, four-year colleges in several sites (Ohio and Pennsylvania) and the 
community colleges in several others (Greater Omaha and Indianapolis, among others) 
argued that gathering performance information related to employment outcomes was 
inconsistent with their educational role. These colleges view their primary mission as 
providing education, often leading to an associates degree or preparing students to 
transfer to four-year institutions, rather than providing job training. As such, they feel 
that a focus on employment outcomes distorts their sense of purpose. 

States and local sites have made efforts to address these concerns in some cases. In 
Texas, for example, local marketing efforts were not successful in persuading many 
community colleges and technical schools to participate in the program beyond initial 
eligibility. The WIA administrative agency had to intervene with the state’s community 
college board to get these public institutions to participate on the eligible provider list for 
the first year of subsequent eligibility. However, these schools have already served 
notice that they may not re-apply in the second year of subsequent eligibility. Similar 
high-level efforts by state officials were mounted in Pennsylvania to persuade 
Pennsylvania State University to participate, as it is a major provider of training with 
satellite campuses throughout the state; no decisions had been made at the time of the site 
visit. In addition, in Pennsylvania, the community colleges in the Three Rivers WIB 
areas are also reluctant to provide performance information. But these schools are locally 
controlled, and the workforce development staff believe that the top city and county 
political leaders will exert sufficient political pressure upon the elected trustees who 
govern the community colleges to get the colleges to participate. 

In some other areas, the community colleges are still participating, even without 
such pressures. These colleges emphasize their economic development role or feel they 
have an obligation to educate or train residents who are not as well served by traditional 
four-year educational institutions. For example, the community college in Macomb-St. 
Clair feels that its community-service obligations outweigh any particular cost-benefit 
analysis that it might apply to ITA requirements. 



Proprietary institutions, on the other hand, expressed few reservations about 
supplying performance information. Those institutions that were heavily dependent on 
public funds were tracking performance information under JTPA and other programs, so 
that compliance with 1TA requirements imposes no additional burden. Other proprietary 
schools that were less dependent on workforce development funds also indicated that 
they are already tracking performance for eligibility for Pell grants, accreditation, or state 
licensing, or because they are strongly market-driven. Others felt that the business from 
1TA was worth the cost of tracking performance. For example, in the Atlanta Regional 
local areas, the New Horizons computer school, a unit of the world’s largest independent 
computer-training school, pursues a corporate strategy to include 1TA and other publicly- 
funded students. 3 School officials believe that JTPA was a good source of revenue and 
its students performed very well. They expect similar results under the ITA system. 

Reshaping the Training Marketplace from JTPA 

Structure of the Market 

The prototypical ITA system seeks to restructure the training marketplace in 
several ways. To begin with, it seeks to dramatically increase the number of vendors 
from which customers can choose, improve quality and reduce costs through competition, 
and drive the system as a whole to better performance outcomes. Additionally, it shifts 
the fundamental market relationship from that between vendor and workforce agency to 
vendor and participant. Under JTPA, training vendors were service providers to the 
Service Delivery Areas and the customer relationship for training vendors was with the 
workforce development agency. While agencies were required to procure those services 
through competitive, arms- length procurements, there was often a close and ongoing 
relationship between agencies and vendors, especially for vendors with many JTPA 
trainees or in areas with few training providers. WIA, in contrast, changes the basic 
relationship. Under an ITA system, the LWIA functions largely as a neutral broker, 
qualifying credentials from vendors who then establish a customer relationship directly 
with the end customer. 

However, while the structural transition in the customer relationship has occurred, 
evidence from the demonstration sites that the ITA system is having any substantial 



3 The level of effort to include publicly -funded trainees, however, varies by location in this 
franchised operation. The school in the Atlanta Regional local areas serves mostly dislocated workers, in 
part because of its location, which is far from low-income neighborhoods. 
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effect on the structure of the marketplace for training at this early stage is limited. As we 
noted in Chapter 111, the customers — mostly low- income adults and dislocated workers — 
are essentially the same as those trained under JTPA. Moreover, since many of the sites 
already eliminated many of their class- size training contracts in favor of individual 
referrals and vouchers, one fundamental element of the structural change — the customer 
relationship — had already occurred. Predominantly, local sites reported that most of their 
vendors from JTPA continue to provide service to the ITA programs. Only a few sites 
reported that they recruited a substantial number of new vendors that might increase 
customer choice. Vendors in the sites that had previous experience with customer choice, 
with one exception, reported that there is little real difference in providing training to 
JTPA and ITA customers. In some cases, our local respondents noted that the 
administrative relationships were precisely the same as they were under JTPA. 

The major difference that vendors are realizing in most sites is that they have fewer 
customers under WIA than they had under JTPA. To some extent, this is a problem of 
timing, because WIA only recently started and few customers have worked through the 
prior levels of services to get to ITA training. In other cases, there is an expectation of a 
permanent reduction in the number of trainees, because some LWIAs expect to have less 
money available for training than previously. Further, there is an expectation that 
effective core and intensive services will be successful in placing some percentage of 
customers in jobs, thus reducing the need for training investments. 

These factors reduce the aggregate demand for training and are thus likely to have 
some adverse effect on the supply of training. These adverse effects are most likely to 
fall on certain proprietary schools and community-based organizations that were most 
dependent on JTPA business. Some of these proprietary institutions are vulnerable to 
disruption under WIA, and even bankruptcy. Several sites noted that some of these 
schools and organizations have already closed or were on the verge of closing. At one 
site, these proprietary schools are responding by modifying their recruitment strategies to 
emphasize private-pay customers. 

Dependency on workforce development funds is even more acute for community- 
based organizations, which often played a substantial role in providing training by 
contract to economically disadvantagpd youth and adults under JTPA. These 
organizations face greater difficulties under the ITA system. First, the presence of the 
ITA as the primary training vehicle reduces tn>. . of contract training at most sites. 
Second, these organizations may have difficulty competing in the ITA marketplace 





because the performance measures for community-based organizations may compare 
unfavorably to other training institutions when customers choose schools. Since none of 
the states indicated that they had immediate plans to adjust performance to account for 
differences in participant characteristics, the CBOs’ emphasis on training lower-skilled 
adults whose outcomes tend be lower than other groups would make them show poorly in 
a simple comparison of performance. 

But even if they surmount the performance issue, these organizations are often 
inadequately capitalized and may have difficulty sustaining operations if they have an 
uneven flow of participants. In the Atlanta Regional local areas, planners noted that 
many community-based organizations lacked adequate capital and were already 
vulnerable to year-to-year shifts in funding under JTPA. Given customer choice in the 
1TA system, they could expect to endure additional periods with reduced cash flow. 
Several had already closed their doors. The results for community-based organizations 
thus appear quite uncertain. 

Tuition Costs 

While costs are relatively less important to individual customers, they are naturally 
quite important to the LWlAs, which, in most cases, have limited funds for training. As 
noted earlier in this chapter, there is little evidence that the ITA system has any impact on 
the tuition that vendors charge. Only one vendor, a truck driving school that was in fact 
quite dependent on ITA customers, reduced its tuition because of the ITA cap. Nor do 
ITAs appear to have any impact on the cost differential between public and private 
schools. Several respondents indicated that their community colleges had always charged 
much lower tuition than the proprietary schools, and the differential is the same under the 
ITA system. On the one hand, the lower tuition charged by community colleges enables 
LWlAs to provide more ITAs from this source with the same pot of money. On the other 
hand, among those LWlAs that provide extensive supportive services, the lower tuition at 
community colleges is offset to some degree by the longer duration of its programs, 
compared to proprietary schools, which implies that supportive services must be provided 
over a longer period of time. A longer duration increases the cost of supportive services 
to the point that, in at least one area, the administrators estimate that there is no net 



difference in the total costs between training an individual at a community college and at 
a proprietary school. 4 

One area is trying to reduce the cost of training from all vendors. The Three Rivers 
W1B is using demonstration grant funds in an effort to increase competition among 
training vendors and drive costs down. Specifically, the W1B is working with a for- 
profit, Internet start-up company that is trying to create a bidding process on a web site so 
that public and private sector organizations seeking training services for their incumbent 
employees could aggregate their requests to help reduce the average cost of training. For 
example, a One-Stop center might need computer network training paid for by an ITA at 
the same time that a small business needs a similar course for one of its employees. The 
web site would bring the two requests togpther and seek a bid from a training provider 
that they expect would be lower than either organization could obtain on its own. The 
demonstration grant investment will help the start-up company test the viability of the 
concept and prepare for investor financing. The Local Board views this activity as a 
mechanism to leverage WIA funds to increase the efficiency of the local market for 
training, regardless of whether funds come from public or private sources. 

Summary 

Congress intended that the ITA system would substantially revamp the training 
system and create a training marketplace. At this point, it is much too early to tell exactly 
what this new system will look like. Nonetheless, we observed a number of persistent 
elements in the relationship between training vendors and the public workforce 
development system in the demonstration sites as the system makes its transition from 
JTPA training to an ITA system. In the first place, because many of the sites had already 
made the transition from class-size training to individual referrals or vouchers, the basic 
relationship between the workforce development agency and the training vendors did not 
change significantly. Second, the demonstration sites are predominantly training the 
same types of participants, dislocated workers and low- income adults, as they did under 
JTPA. Third, at least under initial eligibility, the roster of vendors is largely the same as 
that which existed under JTPA. And finally, many local areas do not have sufficiently 
large labor markets to sustain competition among vendors. 



4 In one state, we found that some vocational training programs at the community college must be 
self-supporting, so their cost structure may not differ as much from proprietary schools as it does where 
there are substantial tax subsidies to the program. 



Alongside the elements of continuity were a number of important changes. With 
the loss of contracts in those places that were still using class-size contract training, 
institutions that were heavily reliant on \wrkforce development funds were facing 
potential trouble, especially if they were not adequately capitalized and able to withstand 
the more variable customer flow under the ITA system. Second, the 1TA system, and 
especially the vendor reaction to its requirement for the submission of performance 
information, exacerbated the differences that long existed between public postsecondary 
institutions — chiefly community colleges — and proprietary schools. Some of the public 
colleges are threatening not to participate in the system under subsequent eligibility, 
either because they are unwilling to bear the cost of tracking certain performance 
information or because they are concerned that their performance outcomes will compare 
unfavorably to other schools that have different student characteristics or are not 
comparable in other ways. Proprietary schools, on the other hand, are very comfortable 
with the performance information requirements. 

Thus, a number of factors are likely to reduce the likelihood of a competitive 
marketplace for training. If the public institutions do not apply to be eligible providers in 
certain areas, their absence will reduce customers’ training options considerably. Where 
public and private institutions now compete, the absence of the public schools would 
substantially dampen choice from a customer perspective. And even if no major players 
drop out, the differences among the various types of vendors may lead to segmented 
market conditions in which little actual competition occurs. 

Nevertheless, it is too early to draw firm conclusions about the likely structure of a 
mature ITA training market because there are still a number of key features of the system 
that remain to be implemented. First, the low enrollment of participants with ITAs at the 
demonstration sites at the time of our site visits made it difficult for our vendor 
respondents to comment on the system. This low number of participants may contribute 
to the reluctance of new vendors to enter the system. Second, with one exception, states 
had not yet entered the period of subsequently eligibility; as such, vendors for the most 
part had not yet been held to performance benchmarks of the sort mandated by WIA and 
did not yet need to comply with the full range of reporting requirements that will 
eventually be necessary. And finally, very few of the states had in place completed and 
fully automated consumer report systems that customers could use as a basic source of 
information. 
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Until these components of the system are fully in place and vendors have had an 
opportunity to react to the full system, it is difficult to know what the final shape of the 
ITA training marketplace will be. This evaluation’s Final Report, to be based on another 
round of site visits that will be conducted in the summer and fall of 2001 , should have 
much to say about the impact of these subsequent system developments. 
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VI. SUMMARY AND PRELIMINARY OBSERVATIONS 



Our site visits to study the ITA/ETP demonstration grantees were conducted at a 
time when many of them were still developing key aspects of their systems. Moreover, 
for a variety of reasons — including fears of funding shortfalls, a strong economy that 
makes job placements relatively easy to obtain, One-Stop centers’ apprehension of 
authorizing training unless it was absolutely necessary, and the need to have customers 
go through core and intensive services before training could be offered — in some sites no 
more than a score of customers had been issued an 1TA, out of hundreds of W1A adult 
and dislocated worker enrollees. Clearly, the systems we saw were for the most part very 
much a work in progress. At the same time, all sites had made substantial progress, both 
in developing policies to serve customers with an 1TA and in establishing an eligible 
provider list and consumer report system. 

Context 

Their progress was facilitated because most were not starting their ITA system 
development from scratch when the ITA/ETP demonstration grants were awarded. In 
fact, almost all had moved sharply away from the exclusive use of contracted training in 
the waning years of JTPA and towards individual referral methods, and over one-half of 
them claimed previous experience with using vouchers for training, either as a CMA 
demonstration grantee or as part of some other pilot program. One-Stop implementation 
grants that they had received from DOL during the mid- to late- 1990s also helped them 
establish the infrastructure that they needed to serve adult and dislocated worker 
customers in a W1A framework. 

Building on this framework, the grantees were using their demonstration funding in 
very different ways, which in some ways reflected the nature and extent of their prior 
progress. Their grant objectives ranged from the very broad (e.g., develop ITA policies, 
build a consumer report system) to the quite specific (e.g., develop a code of ethics for 
vendors). In general, grantees that specified broader goals were not as far along in 
ITA/ETP system development at the time their grant proposals were prepared. By 
contrast, those that specified narrower goals had many elements of their systems already 
in place and were looking to enhance or refine them in some way. 

Grant objectives can also be categorized with respect to their major area of focus. 
The most common cluster of objectives related to efforts to build electronic consumer 





report systems. Again, this was specified in either very broad and sweeping terms or 
with some narrow element of the CSR in mind (e.g., building a customer messaging 
system). Capacity building was another key objective, but again the specifics varied. 

One grantee wanted to hire counseling experts to work with participants and coach case 
managers; another wanted to develop a curriculum for a peer- managed workshop; others 
wanted to develop and deliver training workshops for staff or develop computer modules 
that staff could access as a resource. The third largest category of grant objectives related 
to developing or testing 1TA policies. Again, the specifics varied. One grantee is 
working in conjunction with several other local areas in the state to test alternative ITA 
polices, as a way of gathering information about which ones might be most effective; by 
contrast, another was trying to reduce area-to-area variation in ITA policies by trying to 
facilitate coordination and joint policy development. Finally, some grantees were using 
their grant funds to develop fiscal or tracking software or to automate the training 
provider application process. 

Customers’ UseofITAs 

All of the grantees had embraced the ITA model for providing training services and 
generally seemed enthusiastic about its possibilities for empowering ciBtomers. In fact, 
nearly all of them were planning on using ITAs for training adults and dislocated workers 
almost exclusively; only two expected to make regular use of contracted training for 
meeting the needs of special populations. However, several expected a sharp drop-off in 
the number of persons they would fund for training each year, citing what they felt was 
WLA’s “work first” emphasis and funding limitations caused by their needing to expend 
resources on developing their core and intensive service strategies. 

Regardless of the customer volume that they anticipated, sites needed to develop 
policies to guide the way that customers move through the service levels and make 
training choices. With respect to the first of these, WLA’s emphasis on a hierarchy of 
services mandates that adults and dislocated workers participate in at least one core 
service and one intensive service without having their employment goals met, before 
training services can be considered. The legislation also requires that priority for training 
in the adult program be provided to public assistance recipients and other low- income 
individuals. Understanding how core and intensive services are structured, then, becomes 
key to knowing who accesses the ITA and under what circumstances. 

The case-study sites varied quite a bit with respect to the policies and procedures 
that they established for this purpose. Some noted that customers whom the case 
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manager felt could obviously benefit from training were moved through core and 
intensive services quite quickly. In one case, for example, the core service might be a 
One-Stop orientation and the intensive service the development of the individual 
employment plan. Other sites had more stringent requirements before customers could 
move through to training — for example, by having case managers exhaust all reasonable 
possibilities that the customer might have transferable skills and/or requiring that 
customers spend at least several weeks in core and intensive services engaging in job 
search before training would be considered. Related to this distinction, some sites were 
prone to quickly move customers from self-service to staff-assisted core services, by 
adopting a staff interventionist approach; by contrast, others were much more inclined to 
let customers engage in self-directed job search at their own pace. 

Another key difference across sites related to how intensive services were used as a 
prelude to training. Some sites felt that they could quickly (in core services) identify 
those who would need training services to meet their employment goals. Such 
individuals were placed in intensive services with the full expectation that they would 
shortly undertake training, so intensive services were focused on helping customers 
develop and refine their career and training plans. Other sites felt that intensive services 
represented another opportunity to identify transferable skills and improve job search 
strategies that might obviate the need for training altogether. 

Despite these different general tendencies from one local area to the next, however, 
all the demonstration sites emphasized that guidelines were not meant to be followed 
rigidly and that they adopted a flexible approach to meeting customers’ needs. Thus, in 
almost every site customers could move from core and intensive services to training in a 
matter of a day or two, if circumstances warranted. Similarly, it was pointed out to us 
that dislocated workers could often be more self-sufficient in their job search and career 
planning, while low- income adults often floundered on their own and thus needed staff 
assistance sooner. It seems, then, that a key tenet of WLA that services should be 
customer driven and based on the individual’s own needs appears to be being followed. 

The customer focus is evident as well in the process that sites use to help customers 
make training choices. We identified three models that sites use under various 
circumstances. These are informed choice, which occupies a broad middle ground, and, 
at either extreme, directed choice and free choice. According to the informed choice 
model, One-Stop centers ensure that ti**.. authorized for training receive ample 
information, guidance, and assistance, so that they can make prudent choices with respect 
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to the occupation for which they want to be trained and the vendor who will provide it. 
Within this framework, customers are generally given fairly wide latitude in making their 
own choices. This was by far the predominant mode in the sites we visited. 

Operationally, it meant that customers would be required to undertake a comprehensive 
assessment of their skills and abilities and engage in extensive labor market and other 
research, before an ITA would be issued. Front-line staff play a key role in serving as 
“guides” or “facilitators,” striking what seemed to be an appropriate balance between 
lending the benefit of their expertise while not being overly directive. 

A key element that made the informed choice approach feasible was that 
assessment and research were required parts of the decision-making process. In addition 
to having participants undertake a comprehensive assessment and engage in labor market 
research, which were everywhere required, some sites required participants to conduct 
field research, such as by visiting several vendors and interviewing former trainees and 
employers who hire in that career area. Other sites required that customers attend 
workshops that are either given by case managers or are peer-managed (i.e., to promote a 
team approach to helping customers make choices). Sometimes also customers needed to 
submit a formal application, in which they identify the training field and vendor they 
have chosen and justify their decision on the basis of assessment results and the research 
they have conducted. As a consequence of following these steps to having their ITAs 
approved, customers would come to identify appropriate training choices on their own. 

By contrast, a “directed choice” approach was characterized by the case managers’ 
playing a much more directive role. Only one site used this model predominantly, 
although others would use it under special circumstances, as when customers seemed 
unable to make sense of their assessment results or were reluctant to make judgements 
based on the research they had conducted. In these instances, case managers could be 
quite emphatic in steering customers to the choices that the case manager thought best. 

Finally, the third approach, a “free choice” model, was also used sparingly. 
According to this strategy, case managers would essentially give customers free reign to 
make training choices, so long as the training field was for an occupation in demand and 
the vendor appeared on the ETP list. No site used this approach predominantly. 

However, customers who knew exactly what training they wanted to undertake before 
entering the One-Stop center, and who could justify their choice, often had their request 
honored w.u. little difficulty. 




Given the predominance of the informed choice model, then, our interviews and 
observations lead us to the conclusion that customers are effectively the decision-makers 
almost always. However, their choice is subject to certain limitations established by state 
and local policy. For example, in keeping with the W1A legislation, training can only be 
funded if it is for an occupation in demand. Some local areas met this requirement by 
drawing on lists developed by the state’s labor market information research unit. Other 
sites used locally- developed lists, and a few had no formal lists but instead relied on the 
judgement of the case managers. Typically, where there were such lists, exceptions could 
be made so long as a prospective trainee could present evidence that a job would be 
available once training was complete; however, a few local areas allowed no exceptions 
whatsoever. 

Other restrictions related to dollar or time limits. Nearly all of the sites set a dollar 
cap on the amount of the ITA that would be funded, but these varied widely across sites, 
from a low of $1 ,700 to a high of $1 0,000. Tuition and fees, as well as books, uniform, 
and equipment, would normally be funded by the ITA, and supportive services would be 
provided from a separate pot of money. In keeping with WIA, trainees were typically 
expected to apply for a Pell grant, and amounts they received from that source were often 
applied to the cost of the training, with the ITA paying any balance due. Nearly all sites 
also had time limits on the duration of training that they would support, which they 
usually set at two years. Overwhelmingly, both dollar and time limits were imposed by 
local areas; although they were allowed to impose limits of their own, states generally felt 
that these decisions should be left as a local prerogative. 

Even with these limits, sites could be investing a substantial amount on each 
trainee. For that reason, and also because performance accountability is so central to 
WIA, sites clearly had an interest in doing what they could to ensure that their ITA 
holders completed the training and obtained a well-paying job afterwards. Thus, all sites 
made provisions for keeping abreast of the trainee’s progress and attempted to address 
problems as they arose. Some sites were more proactive than others were, but virtually 
all maintained at least monthly contact with WIA participants in training. 

Given that their performance is publicly displayed as part of the consumer report 
system, vendors also have a clear stake in the trainee’s success, and thus they too played 
a part in monitoring the participant’s progress. Along these lines, proprietary schools — at 
least those that we visited as part of this study — seemed very attentive to students’ needs 
for extra assistance, and were aggressive in helping their students find jobs once the 
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training was completed. By contrast, although community colleges offered counseling 
and placement services, they were typically less proactive in their approach. 

Developing the ETP List and Consumer Report System 

A key element of the training system envisioned by W1A is for there to be clear 
accountability and strong information systems to support customer choice. The eligible 
training provider (ETP) list and consumer report system (CRS) constitute essential tools 
for these purposes. ITA holders can procure training services only from vendors who 
meet the eligibility guidelines imposed by states and local areas for inclusion on the ETP 
list. Information about the vendors’ performance should then be displayed in a consumer 
report system, along with whatever other information that might help support customers 
in their decision making. Developing the ETP list and consumer report system proved to 
be extraordinarily resource intensive. 

A key issue that states grappled with as they assembled the CRS was deciding 
whether only ITA- approved vendors should be included or whether it should include non- 
ITA approved vendors as well. About half of the states adopted each approach. Those 
that were developing a restricted consumer report system emphasized the primary 
objective of supporting training customers in selecting a vendor; those that opted for the 
broader approach were giving priority to developing a resource for the universal customer 
in core services and making the broadest possible use of the resource they were 
developing. Using different logic, both also saw their approach as serving as an 
inducement for vendors to seek ITA eligibility. 

Regardless of the approach they took, sites were generally eager to widely 
publicize the ETP application process and have as many vendors apply for eligibility as 
possible. In some cases, states took the lead role, such as by sending an ETP application 
packet to all state- certified training vendors in the state. In other states, local areas took 
the lead role, such as by communicating with their former JTPA providers or holding 
informational sessions in the community. Most states attempted to automate the 
application process, both to make it easier for vendors to apply and also to expedite the 
state’s and local areas’ roles in processing applications and entering the data into an 
electronic ETP listing. Those states without an electronic application found the process 
substantially more burdensome. 

Only two states, Missouri and Texas, set performance j nents for initial 
eligibility. The others dispensed with such requirements, because they felt ill equipped to 
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make decisions regarding performance benchmarks at such an early stage. In general, 
vendors were not even required to submit performance data as part of their initial 
application. States felt that doing otherwise would impose a substantial burden on 
vendors that they were not yet prepared to meet. 

In keeping with the legislation, the approval process for initial eligibility basically 
worked the same way in all the sites we visited — local areas would first review the 
applications, make a judgement of whether the application should be approved, and then 
pass the application on to the state, along with the local area’s recommendation, for final 
disposition. The one difference was that some states had vendors submit their 
applications directly to the local areas, while other states had vendors submit their 
applications to a central state clearinghouse, which then forwarded the applications to the 
local areas for their review. The latter approach was viewed as easier for vendors, who 
would need to submit just one application for each of its programs rather than multiple 
applications to multiple local areas throughout the state; it also standardized the 
application process somewhat. 

Regardless, a concern that local areas expressed was having a vendor’s application 
for eligibility denied by one local area but approved by an adjacent one. Given that 1TA 
holders can hypothetically choose any vendor on the state list, presumably a trainee could 
select a vendor that the local area funding the ITA had disapproved. Several of the case- 
study sites were attempting to develop a regional approach to ETP review to eliminate 
this possibility. The case managers in another site admitted that they could not envision 
authorizing an ITA if the customer had selected a vendor that the local area had not 
approved, except under exceptional circumstances. 

Another concern was that many states anticipated a sharp drop in the number of 
vendors who were approved once subsequent eligibility began. As mentioned, only two 
states imposed performance requirements for initial eligibility, deliberately with the 
thought that they wanted as many vendors to apply as possible. Similarly, few reporting 
requirements were imposed, beyond asking the vendors to provide basic information 
about each of their programs, such as the duration of training and its costs. But, for 
subsequent eligibility, vendors must be prepared to begin submitting performance 
information about their programs, relating to the completion rates and employment 
outcomes of trainees, including those that were W1A funded and others. Many vendors, 
especially community colleges, have balked at these requirements. They viewed the time 
and effort necessary to assemble the necessary information as not worth the trouble, 
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given that they anticipate serving relatively few W1A- funded trainees. As open- 
enrollment institutions serving diverse community needs, some community colleges also 
feel that their measured performance could misrepresent their actual success, given that 
some of their non-ITA enrollees might lack adequate preparation for training or others 
might not have employment objectives in mind. The only state in our sample that had 
entered subsequent eligibility at the time our site visits were conducted saw an 80% drop 
in vendors’ applications, primarily for these reasons. 

Another reason why the ETP list under subsequent eligibility might be appreciably 
smaller than the initial list is that all states will presumably impose required performance 
benchmarks for subsequent eligibility, while only two states did so for initial eligibility. 
Only four states — Texas, Missouri, Pennsylvania, and Maryland — in our sample had 
decided, even tentatively, what their performance requirements would be for subsequent 
eligibility. Nonetheless, all were grappling with where to strike the balance between 
setting the minimum requirements high enough to reflect their high expectations, while 
not setting them so high as to exclude so many vendors that participant choice is 
seriously compromised. 

Definitional issues were also presenting serious conceptual hurdles. For example, 
even at the point of initial eligibility states needed to make clear what would constitute a 
“program” for purposes of the ETP list. Thereafter, as they attempted to measure 
performance for subsequent eligibility, they needed to decide how key terms would be 
defined, such as who counts as enrolled and what constitutes a completion. Given the 
fact that many community college enrollees may take variable sequences of courses with 
different employment and other objectives in mind, states were generally allowing 
vendors to self-define programs and some were counting as enrollees only those who 
declared their intention to complete the entire sequence of courses that made up the 
program, with employment as the intended objective. 

Amassing the data to measure performance, however the measures were defined, 
also will prove challenging. In keeping with W1A requirements, states were planning on 
relying heavily on using Unemployment Insurance wage records for measuring outcomes 
that were employment related. Thus, vendors would forward the social security numbers 
of enrollees to the relevant state entity, who would then conduct the U1 matching on the 
vendors’ behalf. This approach seemed to make the most sense from the standpoint of 
ensuring completeness, reliability, and comparability. However, the well-known 
limitations of U1 data (e.g., its inability to capture employment in non-covered sectors 
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and, without inter-state agreements, in other states) have led at least two states to allow 
vendors to supplement UI results with employment data from their own sources (e.g., 
surveys). Other states were still weighing this possibility. 

In contrast to employment outcomes, where the states will bear most of the burden 
in data collection on the vendors’ behalf, most states are expecting vendors to supply data 
on their programs’ completion rates. Connecticut, however, is considering computing 
even these rates for vendors, if the vendors will forward to the state the vendors’ 
enrollment database. 

Given the complications that needed to be resolved, at the time of our site visits 
only a few states’ systems had any information about vendors’ performance. Clearly, 
much work remains to be accomplished for the consumer report systems to fulfill their 
function of providing an important resource to guide customer choice. 

Vendors’ Reaction 

In the local areas we studied nearly all training vendors certified as eligible for ETP 
purposes are either community colleges or proprietary schools. These two types of 
institutions have very different missions and define themselves very differently. The 
former have traditionally filled an important role in providing training under JTPA, 
because of the breadth of their offerings and generally low tuition, and it is expected that 
they will be similarly important under W1A. However, many are balking at the eligibility 
requirements that W1A imposes, especially their need to submit performance information 
about their programs. In their view, the low volume of IT A- funded trainees that they 
anticipate does not warrant the time and expense that such a requirement would entail. 
They also fear that their performance would be inaccurately characterized, given the mix 
of customers that many of them serve. Finally, many of them view themselves as 
educational institutions, and not employment and training institutions, and thus believe 
that the CRS’s emphasis on employment outcomes distorts what they see as their true 
purpose. 

By contrast, the proprietary schools whose representatives we met characterized 
themselves as being active in the marketplace and highly performance driven. For their 
own purposes, or to meet other state or federal certification requirements, these schools 
had been accustomed to collecting and reporting performance data and saw no difficulty 
with doing so for ETP purposes. They were also highly adaptable and flexible, 
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modifying course content, starting times, and training durations to better appeal to 
potential trainees. 

Community-based institutions constitute a third group of potential vendors. While 
we did not visit with any community-based organizations that provide ITA training, 

Local Board and One-Stop staff noted that many of these organizations provided training 
to economically disadvantaged adults under JTPA, but are not likely to be successful 
under an ITA system because of their traditionally narrow customer base. In addition, 
their frequent reliance on contract training puts them at risk of losing more customers 
than they might gain in a competitive marketplace. Further, they are usually thinly 
capitalized and are likely to have difficulty coping with an irregular flow ofITA students. 

P REL1MINARY CONCLUSIONS 

The above discussion makes clear that the full ITA and consumer report systems 
envisioned by W1A were still very much under development at the time our site visits 
were conducted. Our upcoming second round of site visits, to be conducted in the 
summer and fall of 2001 , will thus offer the important opportunity to view these systems 
as they have matured. At this point, however, some preliminary conclusions can be 
drawn. 

1 . State and local-area flexibility seems to be embedded in the systems that are 
developing. One of the key tenets of W1A is to devolve decision-making 
authority to the states and local areas, from the federal level. The implications 
of this principle seem clearly to be evidence in the sites we visited. Thus, 
although there are obvious broad similarities in the systems that are developing, 
states and local areas are making unique decisions regarding key features of 
ITA policies. Thus, dollar caps vary widely and the procedures guiding 
customer choice take on a local flavor. Similarly, ETP systems and procedures 
are unique, including with respect to how key terms are defined, what 
performance benchmarks are required of vendors, and what consumer report 
systems look like from the customers’ perspective. 

2. That flexibility, although clearly embraced by states and local areas as a good 
thing, is causing some confusion and uncertainty at this early stage. Thus, 
some local areas are unsure exactly what their policies and systems should look 
like. In light of this uncertainty, more peer-to-peer exchanges would be highly 
valued, so that sites could share ideas and examples. 

3. Overall, there is likely to be a substantial drop-off in the number of persons 
entering training, at least in WlA’s first full year of implementation. This drop- 
off will come about for a variety of reasons, including a strong economy that 
has made job opportunities plentiful, competing priorities for using scarce WIA 
funds, and case managers’ reluctance to authorize training unless it is 
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absolutely necessary. In the coming months, sites will need to reach an 
equilibrium that they think best balances the competing aspects of their 
mission, from providing job search services for the universal customer, on the 
one hand, to providing needed training services to those with poor skills, on the 
other. 

4. Sites are maintaining a strong customer focus in the way they approach WIA’s 
three service levels. Despite the reluctance in some cases to authorize training 
unless it is absolutely necessary, in general we observed that sites are highly 
flexible in their approach to customer services. Thus, although sites have 
guidelines for how customers should move through the service levels, it was 
apparent that those guidelines were not meant to be followed rigidly and that 
customers’ obvious needs were taking precedence. 

5. Informed customer choice is clearly apparent in the way that sites are working 
with customers to help them select training programs and vendors. WIA’s 
emphasis that customer choice should be paramount has clearly been taken to 
heart. Thus, customers are generally given the opportunity to make training 
choices that seem right to them. At the same time, this choice is structured 
within a framework that requires that customers undertake a careful assessment 
of their skills and abilities and conduct extensive labor market and other 
research. To this degree, customers are making choices only after being 
exposed to a range of good information. 

6. Front-line staff are generally playing roles that support informed choice. In 
most local areas that we visited, case managers were playing the role of 
“facilitators,” and were lending the benefit of their expertise without being 
overly directive. To this degree, customer empowerment was being promoted. 
Given that many sites had moved away from contract training in JTPA’s 
waning years, and, in some cases, had previous experience with vouchers, case 
managers often felt that operating under an 1TA system was not that much 
different from what they were accustomed to. 

7. The underdeveloped state of most consumer report systems has meant that firm 
data on vendor performance has not been one of the sources that most 
customers have been using in making their training decisions. Moreover, given 
the difficulties that lie ahead, with respect to data collection and data 
management, it will likely be some time before consumer report systems are 
reasonably populated with information about vendors’ performance. 

8. Incentives for community colleges to participate in the ETP system need to be 
established. Proprietary schools as a whole have been very agreeable to the 
requirements that the eligibility process entails. Community colleges, by 
contrast, view these requirements as not worth the effort, given the few ITA 
customers that they anticipate serving, and their focus as educational 
institutions. Given the important role that these institutions play in giving ITA 
holders meaningful choice, sites need to develop strategies to keep community 
colleges as active players m the training marketplace. 
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9. The ITA/ETP process is presenting substantial challenges to training vendors 
that relied heavily on workforce development funding under JTPA. Reliant for 
so long on contract training for serving special populations, these organizations 
are finding that their customer flow has been gravely interrupted since the 
enactment of W1A. This challenge has fallen especially heavily on community- 
based institutions. Without an alternative customer base, many of these 
institutions are facing insolvency. Their absence will represent a substantial 
loss to their communities. 

10. Processing vendors’ applications and developing the consumer report system 
are extraordinarily difficult and resource- intensive undertakings. In light of 
this, several grantees noted how fortunate they consider themselves to be in 
having been selected to participate in this demonstration, as it has provided 
them with access to special funds for system development that otherwise would 
have needed to come from their regular WIA formula allocation. This 
observation speaks to the difficulty that non- grantee states may be encountering 
with system development, and also for the need for states and local areas to 
develop systems that, once developed, can be sustained at minimal cost. 
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